Integrating resource defence theory with a neural nonapeptide pathway to explain territory-based mating systems

Understanding the causes and consequences of animal behaviour is a fundamental goal in biology [1]. Advances in behavioural ecology, especially during the last three decades of the 20^th century, provide an ultimate-level understanding of how ecological and evolutionary forces shape behaviour and social organization (e.g., [2]). At a more proximate level, advances in neuroendocrinology have increased our understanding of the physiological and molecular mechanisms that mediate changes in behaviour in response to ecological and social stimuli [3–6]. However, a conceptual integration of ultimate- and proximate-level perspectives is yet lacking but may provide a more complete understanding of how neuroendocrine mechanisms mediate variation in behaviour and social organisation [7].

In this review we explore the thesis that neural expression of the nonapeptides arginine vasopressin (AVP), vasotocin (AVT) and their receptor V1a may facilitate territoriality, which in turn may shape mating systems in diverse taxa. First, we provide a framework rooted in resource defence theory to illustrate the relationships among aggression, territoriality, and mating system. Next, we consider the relationship between neural AVP, AVT, and V1a and aggression and territoriality and then gather and assess published studies that have examined AVP, AVT, and V1a variation within particular species. We find that neural vasopressinergic and vasotocinergic action is more closely associated with territoriality than it is with outright aggression. Then, we argue that resource defence theory could explain evolution in a neural vasopressinergic or vasotocinergic circuit at the species level, and we review studies that have compared neural AVP, AVT, and V1a between species that differ in territorial behaviour. Because territoriality is a component of the amalgamation of behaviour that we call mating system, we conclude that selection for up-regulation of a neural vasopressinergic or vasotocinergic circuit could mediate territorial behaviour and therefore play a role in the evolution of different resource-based mating systems.

There are many reasons why animals may behave aggressively, and one of the most common reasons is to defend resources [8] such as food [9], shelter [10], and mates and offspring [11]. In contrast, costs of defence might include injury, vulnerability to predators, time spent not exploiting the resource, and energy expended [12]. The decision to defend a resource depends on benefits - costs, which depends on the resource's distribution in space and time [9], on the individual's motivation and its body condition, and on other ecological factors such as number of competitors, available space, habitat complexity, and predation threat [13–16].

If an individual is attached to a particular site while it is defending a resource, then the space defended is a territory. When resource density is low, the small benefit gained from defending a small amount of resource is not worth the costs incurred from driving away competitors, and individuals may be scattered as they travel long distances seeking resources (Fig.1, far left). As resource density increases, the benefits of maintaining exclusive use of the resource outweigh the costs of defending it, and an individual may behave aggressively, resulting in territoriality (Fig.1, lower threshold). As resources become so plentiful that they can be easily exploited by many, aggressive defence provides no benefit so it is no longer performed (Fig.1, upper threshold). The reaction to competitor density follows a similar pattern: At low competitor densities, individuals are scattered and most of the time there are no competitors nearby to guard against, so aggression is low (Fig.1, far left). When competitor density is higher, an individual may aggressively drive away competitors from a resource in order to maintain exclusive access to it (Fig.1, lower threshold). If space is limited and competitors are forced to remain within the territory as subordinates, then territoriality may take the form of a linear dominance hierarchy [17] or a despotic distribution in which all subordinates are of equal rank [18, 19]. When competitors are very abundant, the costs of defence may exceed the benefits (Fig.1, upper threshold), so an individual may cease behaving aggressively [14] and may abandon a territory and join a group [20, 21]. Such intra-individual plasticity in behaviour predicted from theory has been demonstrated empirically in diverse taxa [15]. Interestingly, territorial relationships may form without outright aggressive acts in many species, due to undisputed differences in competitive ability between individuals, ritualized behaviour, or prior social experience [8].

In many species, territorial behaviour has a strong influence on mating system, defined here as the time of, place of, and partner(s) mated, during mating [2, 22]. Spatial and temporal distribution of resources and competitors may influence territorial behaviour, which then in turn influences mating system. From the male perspective during mating, females are resources and males are competitors. When population density is low, females are scarce or distributed broadly, and individuals may be scattered. Under such conditions, males may use a roaming tactic to find females and promiscuously maximize reproduction (e.g., African striped mouse, Rhabdomys pumilio [23]; Fig. 2, far left). If population density is higher and more competitor males are present, a male may establish a relationship with one female and defend a small territory, resulting in social monogamy (sensu [24]). This seems to be the case in several species of marine fishes ([25, 26]; Fig. 2, center-left). At even higher population density, more females are available. Under these conditions, a male may be able to maintain a larger territory that includes more than one female, resulting in a polygynous mating system, as in the blue head wrasse, Thalassoma bifasciatum ([11]; Fig. 2, center). If population density is yet higher, the number of competing males and the costs of defending are high. A male may only be able to defend a small territory that includes only one female, resulting in monogamy (in some Central American cichlids competition for brood sites is so intense that cooperative defence between the male and female is required to successfully reproduce [27]; Fig. 2, center-right). If mating occurs in a population that is extremely dense, then there are abundant females with which to mate. Under these conditions, the costs of defending one female against the large number of male competitors may not be economical. A territorial male may not be able to exclude male competitors from a territory, and an aggregation may form (Fig. 2, far right). Aggregation is a second type of social organization within which individuals may mate promiscuously. In externally fertilizing species such as butterfly fishes, aggregated individuals may broadcast their gametes simultaneously to maximize fertilisation success [28].

It is important to note that other ecological factors can also influence social organization and mating system. Habitat use may influence distribution (animals aggregating in certain areas of the habitat due to unspecific physical factors like thermal environment, humidity, etc.; like bats in caves). The presence of hetero specific competitors or brood predators may restrict territory size or necessitate parental care [27, 29], and high predation threat may cause individuals to aggregate [30]. For example, zebra finches, Taeniopygia guttata, form dense aggregations but are strongly monogamous [31]; most likely as a result of additional ecological factors that are beyond the scope of resource defence.

Here we present a perspective that extends the concept of intra- and inter-individual plasticity and reaction norm to and beyond the species level. Resource defence theory predicts intra-individual plasticity in behaviour. However, the same ecological factors that may lead to plastic behavioural responses in individuals may also lead to evolution in the tendency to behave territorially. This idea is consistent with recent conceptual arguments that suggest that behaviour may evolve by genetic accommodation, in which the degree of plasticity may not change, but the reaction norm of behavior may change in both effect size and in slope [32, 33]. Population size fluctuates in many species [34], and likely attenuates selection for particular patterns of territorial behaviour. However, within a population, individuals vary both in their ability to plastically adjust behaviour according to local ecological conditions experienced at any given time and in their overall tendencies to behave territorially [35]. Therefore, long term changes in both the stability of ecological conditions and the overall typical state of those conditions could select individuals genetically predisposed to more or less readily adjust their behaviour to local conditions and also to tend to behave either less or more territorially in general. This would result in the evolution of differences in the typical patterns of social organization (e.g., scattered, small territories, large territories, aggregations) exhibited by different species [36]. Indeed, interspecies differences in territoriality are well documented [23]. Furthermore, studies that have found that differences between species in territorial behaviour and mating system may remain after ecological manipulation [37] indicate that in at least some species these differences are genetically selected and not simply plastic responses to local ecological conditions (see also [38]).

One of the best studied functions of AVP and V1a has involved a particular circuit in the brain that originates in the bed nucleus of the stria terminalis (BNST) and the medial amygdala and projects to the lateral septum (Table 2; [45–47]). AVP is produced in the cell bodies of neurons in the BNST and the medial amygdala. The axons of these neurons extend to the lateral septum where V1a receptors are found on their postsynaptic targets (Fig. 3A). In males of the monogamous prairie vole, Microtus ochrogaster, experiments using pharmacological methods and transgenic animals have demonstrated a causal relationship between AVP and V1a in these brain regions and behaviour patterns generally considered characteristic of monogamy, such as mate affiliation, nest defence, and paternal care for offspring [48–51]. The lateral septum also receives AVP from adjacent regions in a paracrine fashion. However, the simultaneous elevation of AVP in the BNST and V1a in the lateral septum when comparing sexes or species with different mating systems [52, 53] suggest that this circuit plays an important role in all of these different behaviour patterns. An important question is how AVP and V1a expression can be associated with both affiliative behaviour and with aggressive defence of a territory, which appear to be very different forms of behaviour [54].

Brain region	Description
Bed Nucleus of the Stria Terminalis (BNST)	Contains cell bodies of AVP- or AVT-producing neurons in tetrapods.
Medial amygdala	Contains cell bodies of AVP- or AVT-producing neurons in tetrapods.
Supracommissural nucleus of the ventral telencephalon (Vs)	Putative teleost homolog of the medial amygdala and BNST, but possesses no AVT-producing cell bodies.
Preoptic Area (POA)	A neuroendocrine integration centre located at the interface of the hypothalamus and telencephalon. Contains groups of AVP- or AVT-producing neuron cell bodies that project axons throughout the brain.
Paraventricular nucleus (parvocellular subdivision in teleosts)	Associated with stress and subordinate behaviour. In teleosts, part of the POA.
Supraoptic nucleus (magno-/gigantocellular subdivision in teleosts)	Associated with aggression and reproductive behaviour. In teleosts, part of the POA.
Lateral septum (LS)	In tetrapods, a collection of neurons in the medial forebrain lying generally anterior to the anterior commissure. Receives axons of AVP- or AVT-producing neurons. Associated with several types of social behaviour.
Ventral nucleus of the ventral telencephalon (Vv)	Putative teleost homolog of the lateral septum.
Ventral pallidum (VP)	A brain region immediately ventral to the lateral septum in tetrapods. Receives axons of AVP- or AVT-producing neurons. Implicated in social pair-bonding in rodents.

One possible explanation for the septal AVP-V1a circuit's stimulatory effect on both affiliation and aggression could be that it more generally enhances social cognition and in doing so stimulates the formation all types of social relationships [55]. Here we define social cognition specifically as the acquisition and retention of social information [56]. Territoriality would seem to require establishing relationships with other individuals (albeit a confrontational context) and the recognition of individuals upon a subsequent encounter. Social cognition may thus be necessary to form and maintain relationships with both mates and offspring and with competitors (such as familiar neighbours or “dear enemies”) when maintaining a territory, compared with simply roaming over a broad home range [44, 57–59]. Additionally, holders of large territories are probably continually establishing new relationships as neighbouring individuals change, compared to holders of small territories who have fewer neighbours, or compared to roamers who may encounter many individuals but never form a relationship. Therefore, up-regulation of a neural circuit that stimulates social cognition, specifically the establishment and maintenance of social relationships in specific surroundings, could facilitate both affiliative and aggressive behaviour, depending on social context.

In the sections below, we review studies that have measured aspects of neural AVP, AVT, and/or V1a. The studies are of two general designs: they either measure and compare neural AVP, AVT, and/or V1a between different behavioural phenotypes or they manipulate neural AVP, AVT, and/or V1a and measure the resulting behaviour. Studies that measured neural AVP, AVT, and/or V1a have done so using a variety of different methods. Many studies used immunohistochemistry (e.g., [60]), a laboratory technique used to anatomically visualize the location of a specific protein by using a specific antibody that binds to it. The antibody is visible with a fluorescence microscope when it is bound to a special light-emitting molecule. These studies typically then compare the sizes or numbers of AVP or AVT-immune-reactive (AVP-ir) cells or density of staining within the cells. Some studies measure c-Fos expression in a similar way in addition to the AVP- or AVT-ir (e.g., [61]). Immediate early genes, such as c-Fos, are rapidly and transiently upregulated following an external stimulus. Therefore, increased expression of both AVP or AVT and c-Fos suggest that an AVP- or AVT-expressing neuron is increasing activity in response to a stimulus. Other studies have measured AVP or AVT mRNA. In situ hybridization binds mRNA using a radioactively labelled complementary strand of DNA or RNA (e.g., [62]). Quantitative PCR measures AVP or AVT mRNA by binding it with a light-emitting molecule (e.g., [63]). Finally, a few studies have measured AVT directly using high-performance liquid chromatography; a new application of this method (e.g., [64]). V1a receptors have been identified using autoradiography: binding to a radioactively labelled ligand (e.g., [65]). Many studies that have manipulated neural AVP, AVT, and/or V1a have directly applied AVP, AVT, or V1a antagonist. This has been done by either infusing AVP or V1a antagonist into one of the brain ventricles, (e.g., [66]) or directly injecting AVP or AVT into a specific brain region or into the intraperitoneal cavity (e.g., [67]). A few studies have “knocked down” AVT production in a particular brain region by injecting antisense oligonucleotides that bind to the target mRNA and prevent it from being translated [68].

A substantial number of studies have examined neural AVP, AVT, and V1a within and among individuals of a particular species. In this section, we review those studies to asses if neural AVP, AVT, or V1a is closely associated with either aggression or territoriality. An association with aggression would indicate a direct relationship between the molecular pathway and aggression. An association with territoriality would suggest that the pathway has a cognitive function, which would better explain the perplexing diversity of behaviours that have been associated with AVP, AVT, and V1a in previous studies. Furthermore, if neural AVP, AVT, or V1a is associated with territoriality within species, then it may also be possible for neural AVP, AVT, and V1a to also be associated with territoriality between species.

Below we review studies that have compared aspects of neural AVP, AVT, or V1a in two or more species that differ in aggression, territoriality, and mating systems and interpret them in the context of resource defence theory. On an inverted-U resource defence function we plot relative expression of neural AVP, AVT, and V1a, using social organization (e.g, scattered, large territories, aggregations) as a proxy for resource density and competitor density, then we evaluate congruency between patterns of neural expression and the species-typical level of territorial behavior (no territories, small territories, large territories) (Fig. 2). Just as our analysis assumes that species-typical patterns of social organization are selected in response to species-typical ecological conditions and population densities, it also assumes that neural AVP, AVT, and V1a are consistent enough within species to persist under the artificial social conditions elicited by a laboratory environment. Indeed, the statistically significant differences between species reported in numerous published studies (see below) suggest that neural expression within a species is consistent enough to persist under artificial socio-ecological conditions (sensu [61]). Many of the studies discussed below are two-species comparisons, which are limited in their ability to allow conclusions about the evolution of traits [130]. However, multiple observations of a particular pattern more strongly indicate that the pattern is a result of evolutionary change and not merely a random difference between species [130].

The results of our analysis indicate that neural vasopressinergic and vasotocinergic action facilitates territoriality, which in turn may shape mating systems. We found that, within species, neural AVP, AVT, and V1a are often associated with territorial behaviour. Similarly, among species, we found that neural AVP, AVT, and V1a are associated with territorial behaviour in a pattern consistent with resource defence theory. This suggests that ecological pressures may, through natural selection, drive up- or down-regulation of expression in a particular neural circuit that stimulates territorial behaviour, resulting in the evolution of differences in reaction norms between species. Because territorial behaviour can influence the acquisition of mates, then such a neural circuit may also influence the evolution of mating systems.

One possible reason that neural AVP, AVT, or V1a expression might be higher in territorial individuals may be because it directly stimulates outright aggressive behaviour (reviewed by [70]). However, several studies indicate that AVP, AVT, and V1a do not necessarily stimulate aggressive behaviour but instead initiate an aggressive disposition, which might be beneficial during the establishment of a social relationship. Although male prairie voles ICV-infused with a V1a antagonist failed to develop a typical increase in aggressive behaviour after co-habitation with a female, aggression was not reduced in males that had previously-established pair-bonds with females [48]. Decreases in aggression caused by nonapeptide antagonists in birds and fishes were also restricted to occurring during establishment of a relationship but not after a relationship had formed [83, 108]. Furthermore, although Oldfield et al. [150] found neural V1a2 expression to be associated with territory size in two species of cichlid fishes, raw numbers of aggressive acts compared between territorial males of each species were negatively associated with V1a2 [38]. Therefore, although neural AVP, AVT, and V1a may stimulate aggression in mammals, birds, and fishes, it seems to do so primarily during the formation of social relationships and not necessarily in other contexts.

The role of neural AVP, AVT, and V1a in the formation of social relationships could be related to its known role in general social cognition. AVP and V1a in the lateral septum are well known to facilitate social memory [55]. For example, in rats, V1a agonists injected into the lateral septum facilitate, but V1a antagonists and AVP anti-sense DNA inhibit, the memory of same-sex conspecifics and familiar juveniles [151]. The association described above between AVT and V1a in the septal circuit and gregariousness in finches is consistent with this idea – the tendency of gregarious animals to approach other individuals and to choose to associate with large groups over small groups is similar to the tendency of animals with large territories (compared to animals with small territories or roaming-tactic species) to establish social relationships [152]. Thompson & Walton [153] found that AVT ICV-administered to highly social individual male goldfish, Carassius auratus, inhibited social approach toward same-sex conspecifics, but treatment with an AVT receptor antagonist stimulated social approach [153], but this mechanism was later found to be mediated by a different circuit in the hindbrain [154]. Therefore, neural AVP, AVT, and V1a, at least in one circuit in the forebrain, seems to be responsible for generally positive-valence behaviour important for establishing social relationships, which may include aggression, but also territoriality (independent of aggression), and gregariousness.

Ophir [5] proposed a two-axis interaction between social bonding and social spacing that determines overall social behaviour, such that seemingly contradictory aspects of social behaviour may in fact arise from the coordinated action of multiple vasopressinergic circuits in the brain (see also [3, 96]). The finding that V1a receptors are widely distributed throughout the vertebrate brain [155–158] and that this expression pattern is highly conserved across vertebrates [159], is consistent with the notion that AVP (or AVT) and V1a expression in different brain regions likely mediates different behavioural processes such as social bonding and territoriality in a manner that is similar across lineages [3, 6, 94, 96, 148, 160]. The septal vasopressinergic or vasotocinergic circuit could variably stimulate additional downstream circuits, each of which are more directly responsible for either affiliative or aggressive behaviour. Alternatively, there may be subdivisions within the septal vasopressinergic or vasotocinergic circuit that are differentially responsible for stimulating different forms of social behaviour [161].

We also found that, at least in mammals and fishes, differences between species’ mating systems are consistent with differences in neural AVP, AVT, and V1a expression. Despite the apparent connection with monogamy in prairie voles [48], across different vole species the association between the septal vasopressinergic circuit and monogamy has not appeared to be very robust: the higher abundance of V1a in the lateral septum in M. montanus than in M. ochrogaster [65, 135, 136] had previously been interpreted to be inconsistent with the idea that the septal vasopressinergic circuit was associated with mating system across species [3, 5]. To reconcile these disparate results, Goodson and Bass [162] proposed the idea that V1a expression in the lateral septum of cricetid rodents is consistent with the patterns of space use observed in different mating systems (i.e., home range size and overlap), but not with pair bonding. The current work expands this idea by showing how stimulation of a neural vasopressinergic or vasotocinergiccircuit could enhance social cognition and the formation of social relationships, thereby facilitating territorial ownership and influencing mating systems in those species in which males may defend resources in order to mate. It is important to note, however, that these factors do not necessarily interact in this way to determine mating system in all species. In some species, other ecological factors (such as high predation threat or need for parental care) may play a key role in shaping social organization and mating system. In estrildid finches, overall social organization is dissociated from mating system. Species may exhibit territorial or aggregating social organizations, but all are monogamous. Thus it is not surprising that septal circuit AVT and V1a do not predict territoriality or mating system in these species.

Although the septal vasopressinergic or vasotocinergic circuit has clearly been found to be responsible for stimulating affiliative and gregarious behaviour, pair bonding in male Microtus spp. is likely mediated not by the lateral septum but by the ventral pallidum, a region immediately ventral to the lateral septum [163]. Although males of the polygynous M. montanus have higher V1a densities in the lateral septum than do monogamous male M. ochrogaster, the latter have higher densities of V1a in the ventral pallidum [164]. Blocking the binding sites of V1a in the ventral pallidum prevented mating-induced preference for a particular female partner in male M. ochrogaster [163], and experimental induction of V1a receptor expression in the ventral pallidum of M. pennsylvanicus dramatically increased partner-preference in this normally promiscuous species [164]. Young & Wang [163] proposed that pair bonding in M. ochrogaster is caused by an interaction between AVP in the ventral pallidum that causes partner preference and AVP in the lateral septum that causes partner recognition (in combination with dopamine in the nucleus accumbens). A teleost fish homolog of the mammalian ventral pallidum has not yet been conclusively identified [80], but a recent developmental and neurochemical study suggests that it may be the caudal portion of area Vv [165]. Additionally, recent evidence has found that monogamous male prairie voles expressed higher OT receptor density in the nucleus accumbens than did non-monogamous males [166], a pattern proposed previously to be important to elicit mating-induced partner preference in females [163]. It will thus be fruitful to examine whether a similar pair-bonding circuit may be present in other vertebrate taxa, and whether it involves AVP, AVT, or OT or isotocin [167].

To fully understand the mechanisms underlying different mating systems, more data are needed from females. Female behaviour plays an important role in shaping mating systems. In many species females maintain territories that may or may not be congruent with male territories (see examples provided above for cricetid rodents), and mating systems emerge from conflict between male and female interests [168]. Unfortunately, we have not found published data on the relationship between nonapeptides and territorial behaviour in females. It seems likely that a neural mechanism underlying space use in females might involve OT, which has been found to vary with social structure in tuco-tucos [169]. In fact, OT has recently been found to activate V1a receptor and weakly stimulate flank marking in male Syrian hamsters, so it may play a role in shaping mating systems in males as well [170].

In sum, we have identified an association between a proximate-level neuro-endocrine mechanism and an ultimate-level behaviour pattern that has a direct effect on the social organization of populations. Specifically, neural AVP, AVT, and V1a are associated with territorial behaviour in males, and the most parsimonious explanation for the role of AVP, AVT, and V1a in territorial behaviour is to stimulate social cognition, not necessarily aggression. Furthermore, we have applied the concept of intra- and inter-individual plasticity and reaction norm to evolutionary patterns at and above the species level. Because mating in some species involves territorial behaviour, neural AVP, AVT, and V1a also seem to mediate mating system in both mammals and teleost fishes, but not in the bird species studied to date. Territorial behaviour is most likely one key trait driving resource defence-based mating systems, whereas mating systems are emergent properties arising from the presence or absence of territorial behaviour, pair bonding behaviour, parental behaviour, and perhaps other, less obvious, types of social behaviour. Each one of these types of behaviour is most likely stimulated by one particular (nonapeptide) neural circuit. We expect that interdisciplinary integration of ultimate- and proximate-level perspectives will continue to improve our understanding of social behaviour in the future [7]. Additional comparative studies that include a detailed understanding of social organisation, isolate a single aspect of behaviour, such as pair bonding, and employ a fine resolution of brain anatomy have a promising outlook for identifying discrete nonapeptide circuits and characterising their particular contributions to larger, emergent forms of social behaviour.