How assessment and reflection relate to more effective learning in adaptive management

How to cite this article: Biggs, H., Breen, C., Slotow, R., Freitag, S. & Hockings, M., 2011, ‘How assessment and reflection relate to more effective learning in adaptive management’, Koedoe 53(2), Art. #1001, 13 pages. doi:10.4102/koedoe. v53i2.1001 Assessment (an immediate evaluation of significance or performance) and reflection (a lengthy, deep consideration) should be important components of adaptive management leading to learning. In this paper we use a prototype adaptive cycle and feedback framework, which are related to some aspects of learning theory, to examine the extent to which assessment and reflection were applied in a series of studies and initiatives in the Kruger National Park. In addition to evaluating assessment and reflection, we also considered how the various contributing components of each case were inter-related to provide a holistic view of each initiative.


Introduction
South African National Parks (SANParks) formally adopted an adaptive management paradigm (Walters 1986) in the form of strategic adaptive management (SAM) (Roux & Foxcroft 2011) in the Kruger National Park (KNP) in the 1990s.The park has become widely known for this initiative, which has now also spread to other protected areas and other natural resource management situations (Freitag, Biggs & Breen, in review).It is therefore fitting to determine the extent to which adaptive management performance in the KNP has been assessed and/or reflected upon, and how this has contributed to further learning in a management context that typically embraces spatial and temporal variation and socioecological linkages.This paper examines whether adaptive management, designed for decision making in uncertain environments, is being implemented in a rote, mechanistic way or supported with effective assessment, reflection and learning.
Evidence-based policy and decision making processes and the theory of learning are used in a wide range of fields (e.g.healthcare, see Gray [1997]).It is reasonable and more generic to see the adaptive management processes discussed in this paper as a particular (albeit more unpacked and iterative) form of the construct called the 'policy cycle', as exemplified by the wellcited Australian Policy Cycle (Bridgman & Davis 2004).Although the latter has been criticised for simplicity and linearity, broad parallels with several of the sequential steps of adaptive management emerge (Roux & Foxcroft 2011).It is likely that each has drawn on the other's underlying thought processes and may continue to do so.From an examination of fundamental references on learning (e.g.Kolb 1984;Schön 1983) it is clear that such processes are linked to learning; indeed, individuals and organisations adapt (and therefore often survive or prosper) based on re-conceptualisation following reflective observation of past experiences.It is therefore appropriate that this paper frames learning as the central goal of assessment and reflection in adaptive management.After examining assessment and reflection specifically, we draw directly on other learning work (the interviews for which were conducted on KNP staff) to examine the relationship between six learning dualities (Roux, Murray & Van Wyk 2008) and assessment and reflection, respectively Assessment generally means to 'evaluate or estimate the nature, quality, ability, extent, or significance of' whilst reflection is viewed as 'a calm, lengthy, intent consideration' (wordnetweb.princeton.edu/perl/webwn).Although the words 'assessment' and 'reflection' are sometimes used interchangeably in the conservation literature, we shall use the definitions as described above (recognising limited overlap), except when citing.In the context of adaptive management, assessment is commonly intuitively thought to lead to reflection.Such reflective behaviour should specifically be carried out in the full context of the system under consideration, observing and modifying its own attributes and behaviour, with a view to learn for future action.An organisation that practices such reflective behaviour can be considered an adaptive learning organisation (Garvin 1993) in which creating opportunities for reflection through a process of consistent and structured review of past performance can be a critical factor.Argouridas and Race (2007) have argued for the importance of evidence-based reflection in learning and knowledge management within higher education.However, too narrow a focus on (often limited) evidence may also hide the controlling influence of context.Such reflective aspects are widely underemphasised, often as a result of an incentive to progress with implementation and so be seen as 'doing something' (Allan & Curtis 2005) or because the organisational culture does not promote learning and adaptation (Fazey & Schultz 2009).It is therefore important to consider the prominence given to reflection in the broader learning literature (Schön 1983).
Literature regarding effective management of protected areas describe assessment and evaluation cues in a sequential, iterative and adaptive process at different points in the management life cycle (Figure 1), which is itself a version of an adaptive cycle (described as the adaptive review cycle of SAM by Rogers [2005]).Hockings, Stolton and Dudley (2004) emphasise the need to assess and evaluate continuously during the cycle, rather than only after the outcomes step (Figure 1).We specifically avoid the term 'evaluation' onwards, except where explicitly used by others (e.g.Hockings et al. 2004).We regard 'evaluation' to be closer in meaning to 'assessment', although it can include the important additional aspect of passing judgement, which suggests some reflection, according to our definition.Experience and learning provide improved understanding of the complexity and changing context, which, in turn, allows for refinement or revision of different steps in the cycle.By following all six evaluation cues (the straight solid arrows radiating from the centre) and considering them in relation to one another participants can view the overall process holistically, for example, as applies to park management (Hockings et al. 2009).Also, it is regarded to be diagnostically useful to 'step back through the process' so that each component can be considered with regard to its preceding input.If the actions represented by feedback arrows in Figure 1 are ignored, assessment and/or reflection will not occur in appropriate contexts, and hence opportunities for meaningful and holistic learning will be missed.If assessment and reflection are considered to be fundamentally distinct processes, lumping them together may lead to practitioners subsequently acting without one or the other consciously having taken place.This may lead to poorly considered action in a complex, changing setting such as management of a park or natural resources and negate learning opportunities or effectiveness.
Another characteristic depiction of SAM is a series of welldefined nested feedbacks as shown in Figure 2a.('Nested' means feedback is contained conceptually within another feedback, in a logical sequence.)Such nested feedbacks are usually shown most usefully as arising from the postoperationalisation learning opportunity of the cycle (i.e.once things on the ground start happening), but they could be drawn from any step in the cycle (as shown in Figure 2b).Indeed, the recommended diagnostic approach of sequentially stepping backward, as shown in Figure 1, can be considered nested.In the context of the process steps of SAM (Rogers 2005) there are several opportunities for 'advance reflection' or 'stepping forward' in the early forward planning components, specifically when predicting consequences of potential management options or the societal acceptability of these, before the specific management action is selected.Such nesting for attempted prediction or scenario planning occurs through a process of 'thought experimentation' (Figure 2b) or even explicit modelling (Starfield & Bleloch 1991).Figure 2 contains a key decision point ('select options') and although other decisions are taken at other points, this decision point gives rise to the most direct 'action on the ground'.An analogous point can be inferred between 'process' and 'outputs' in Figure 1.A combination of Figures 1 and 2 hence clarifies opportunities to facilitate structured forms of both assessment and reflection.Nevertheless, we need not be bound by only these pathways, as additional unstructured assessment (i.e.without any obvious direct reference to particular heuristics depicted in Figures 1 and  2) and particularly unstructured reflection, may add further value.
Checking that each nested step is being performed constitutes one way of verifying that adaptive management is indeed being practiced, or that it is at least procedurally sound.Yet the context in which different situations are assessed or reflected upon varies a great deal and usually changes over time, which makes iterations, including explicit reflection on the changing context itself, essential.Furthermore, the subcontexts (e.g.fire management, surface water source management, etc.) that constitute any particular context (e.g. the management of the park as a whole) change differentially and it is the aim of management actions to alter some of these subcontexts.As the context becomes more confined, management influence increases.However, as the context grows and becomes more related to governance, management influence usually decreases (Nkhata & Breen 2010).The appropriate scale(s) and/or organisational levels at which an assessment or reflection initiative should focus should thus be considered.Levels often correspond to the different levels of park objectives.As such, scale of selection may determine whether different contexts or subcontexts are appropriately and fairly assessed and reflected upon and,  hence, whether these have been wisely chosen as a basis for thinking about management action.As one proceeds along the cycle in Figure 1 (from planning onwards), the scale of reflection usually increases in both space and time.There is a move from specific objectives, as defined through planning and implementation, to more general goals as the process approaches the scale of re-visioning.Accordingly, there is a move away from considering only some parts at particular scales towards a more holistic, integrated and systems-based consideration.If processes are institutionally embedded and aligned with the context, within an enabling organisational culture (see Growcock, Sutherland & Stathis 2009;Stirzaker, Roux & Biggs 2011), adaptive management can be improved.
On the other hand, organisations typically have several procedures and approaches, many of which are not adaptive in nature (and are even designed not to be), and it can be challenging to dovetail these with adaptive assessment and reflection.
In addition to the adaptive processes outlined earlier (and depicted graphically in Figures 1 and 2), this paragraph provides a brief history of adaptive management in KNP as a backdrop for understanding where assessment and reflection, as described in this paper, fit in.Mabunda, Pienaar and Verhoef (2003) described, before the advent of explicit adaptive management in KNP in the 1990s, a series of different management eras, each with a particular style.These eras set favourable precursor conditions (such as regular interaction between scientists and managers and joint responsibility) for explicit adaptive management to be accepted once the opportunity arose (see Biggs & Rogers 2003, Box 1).Freitag et al. (in review) describe how adaptive management practice, after expanding from the KNP Rivers Research Programme to the general management of the park, spread to influence other aspects of conservation management in South Africa during the first decade of this century.Examples include drafting guidelines for protected areas and catchment management and planning conservation initiatives, which co-evolved with evaluations of park effectiveness (Belokurov et al. 2009) and reporting of the state of biodiversity (Knight 2008), both of which had a culture of assessment.

Methods
We reviewed as many earlier initiatives that had focused on the operation or performance of adaptive management in the KNP as possible.These were mostly in qualitative or narrative form, although some had semiquantitative 'scores'.Some of the initiatives were internal (part of the prescribed procedures in the adaptive management protocols), whilst others had been subjected to previous external review (as one would expect in an independent audit).However, this distinction actually translates to a continuum spanning 'designed as internal' to 'initially mainly external', because (1) feedbacks are intrinsic to adaptive management and contribute to defining a wider system boundary and (2) implementation of the initiatives often involved the same people who initially advised on the operation of the system, possibly owing to their existing knowledge.
These initiatives were subjectively rated for their emphasis on assessment, reflection and inter-relatedness, independent of impression of the study quality (Table 1).Inter-relatedness pertains to the holistic approach of the initiative and considered how the different levels of adaptive feedbacks are related and considered jointly, given that they can be viewed as nested within one another.We regarded interrelatedness to enhance assessment and reflection, thereby facilitating more effective learning.We scored the respective columns in Table 1 based on our impressions of the extent to which each initiative dealt with assessment, reflection and inter-relatedness.We did not consider the interpretations of how these aspects contributed to the study system, but simply whether the aspects could have been noticed and discussed.Cross-correlation results we report in Table 2 may therefore partly reflect differential study designs of the various initiatives.However, our interim conclusions of this initial analysis include conclusions based on actual findings of particular studies (as in a literature review), shown as numbered references in Table 1.
Following this initial analysis, which serves as a potential bridge to the issue of learning, we describe two further studies that dealt explicitly with learning in the KNP in or close to the context of ecosystem management.One of these studies produced a set of dualities, which we thought could support our own exploration into assessment and reflection.
The third leg of the method involved mapping examples of assessment and reflection in adaptive ecosystem management in the KNP onto these paired strategy options (Roux et al. 2008) or dualities (Table 3) and then exploring how assessment and reflection appeared to be related to each other and to learning, respectively (Table 4).The dualities are presented as sharply dichotomous (binary), whereas they can, in reality form more of a continuum in some of the cases (e.g.learning cf.unlearning).

Initial analysis of how adaptive management has been assessed and reflected upon
The KNP has explicitly attempted to implement strategic adaptive management, at least in the sphere of ecosystem management, for 15 years, and the methods and intentions have been documented in publications, reports and brochures (e.g.Biggs & Rogers 2003;IUCN Freshwater Task Force and Skukuza Group [in prep.];Pollard & Du Toit 2005;SANParks 2008).This has attracted the attention of scholars and managers, with several initiatives or studies aimed at meaningfully looking directly or indirectly at this drive (Table 1).
The studies and initiatives listed in Table 1 provide a useful idea of how well (1) the defined adaptive management processes, and/or (2) actual management outcomes (the ultimate goals of SAM) have performed.These are invariably presented in narrative form in the originals, although some do include semiquantitative scores.Our general summary Study numbers in first column are as referred to in the supporting text in the discussion and in Tables 3 and 4. AM, adaptive management; METT, management effectiveness tracking tool; SAM, strategic adaptive management; TPC, threshold of potential concern (defined by Biggs, H.C. & Rogers, K.M., 2003, 'An adaptive system to link science, monitoring and management in practice', in J. du Toit, K.M. Rogers and H.C. Biggs (eds.),The Kruger Experience: Ecology and management of savanna heterogeneity, pp.59−80, Island Press, Covelo, as ecological or management endpoints that trigger action); KNP, Kruger National Park; KZN, KwaZulu-Natal. a , rating scale: 0, no emphasis; 1, minor emphasis; 2, fair emphasis; 3, strong emphasis.b , indirect assessment, as the initiative was not targeted specifically at adaptive management.
after reviewing the findings of these studies follows below.
Numbers in square brackets refer to the studies listed in Table 1 that most clearly support each point: • Adaptive planning has progressed and produced a more shared rationale (i.e. more overlap in underlying mental models) amongst most key stakeholder groups, seemingly greater than we can perceive prior to this from documented history (over 100 years Any possible significant cross-correlations which may arise with the use of greater sample sizes in future, may also reflect differential study design of the various initiatives.This could be considered equally informative, in the sense that the overall system of study and action is in some cases less geared to picking up on or producing, say, actual reflection.

Moving towards improved learning
Our fundamental premise is that assessment, reflection and their thorough inter-relation create a foundation of shared understanding (amongst all the relevant stakeholders associated with a national park) that directs and shapes deliberations and creates the basis for adaptive learning.How can we then assess whether this learning, in the context of adaptive management, takes place effectively?A learning organisation (Garvin 1993) is one skilled at creating, acquiring, and transferring knowledge and is able to modify its behaviour to reflect new knowledge and insights.Schön (1983) provides a theoretical understanding of learning and Michael (1995) specifically lists criteria to assess learning organisations, illustrated by questions such as whether (1) error is accepted, and fear of error is offset, in participating individuals, (2) group or social learning takes place, with facilitation rather than chairing of meetings, and (3) crises are leveraged as opportunities for learning.These kinds of question are mirrored by broadly similar ones for benchmarking adaptive management (Stankey, Clark & Bormann 2005), determining effective environmental administration (Paehlke & Torgerson 1990), assessing resilience and sustainability of socioecological systems (Walker & Salt 2006) and managing businesses sufficiently 'mindfully' to sense emergent signals and their meaning in a currently uncertain and complex world (Weick & Sutcliffe 2001).It would thus appear that these fields are all strongly influenced by the way learning is enabled in individuals and across organisations and by whether or not behaviours have been modified in response to growth in understanding as per the reflection stage of the experiential learning cycle (Kolb 1984).A recent consultation delivered to SANParks Scientific Services staff (Narrative Lab 2009) deals with factors enabling performance in the SANParks work environment, which staff consider to be complex and dynamic, after having worked through responses to questions of a similar genre to these above.Key recommendations, such as more regular use of probes (small, adaptive experiments, mainly in the organisational milieu), to help structure learning emerged.
KNP staff were interviewed in two studies to probe their learning experiences after the inception of adaptive management.As part of an IUCN study, Pollard andDu Toit (2005, 2007) interviewed rangers whose sections included perennial rivers to establish their attitudes towards the adaptive management of rivers.The results showed good, but not universal, uptake of river objectives amongst participants and an understanding that their own inputs guided these objectives as part of a holistic vision (S.Pollard, pers. comm.).Rangers were aware of, and to an extent also participating in influencing, the changing context (e.g.cultural, legal, climatic and water use context).Long-serving

TABLE 3:
The two facets of each learning duality as aligned with assessment and reflection.Each case is related to adaptive processes through pointing to phases in Figure 1 (context, planning, inputs, process, outputs, and outcomes) and process linkages in Figure 2. (The phrase 'bottom loops' refers to loops marked 1-3 and 'upper loops' to those marked 4-7 in Figure 2a).All themes used here are discussed in detail in one or more of the references given in Table 1.

Fire management
Fine-tuning an operational threshold, such as adjustment of the calibrations of burnt areas used in monitoring to guide fire management, as better data or histories became available.Influenced mainly bottom loops, and is process and output oriented.
Underlying assumption of the heterogeneity-based, integrated fire policy was that biodiversity was likely to be enhanced or maintained if patch burns were used instead of conventional and safer circumference ignitions.Reflection resulted in this being explicitly tested in a major landscape experiment; provocative findings are emerging, which may challenge overall notions of heterogeneity in the organisational vision; influenced upper-loop processes and full cycle in Figure 2.

Practicing conceptual and operational learning Associated with practical learning Mainly conceptual learning
Threatened species management Learning related to management and monitoring of rare and threatened species such as roan antelope, black rhino and African wild dog, based partly on species biology studies and partly on day-to-day experience; mainly focused in bottom loops and process-output-outcome.
Threatened species strategies can come into contention with ecosystem management goals; there are too many individual threatened species for meaningful attention to be given to each (Rebelo et al. 2011), which is a key result of conceptual learning.Reflection thus produced a rare species framework, which addresses loop 4, to achieve the overall vision in loop 6. Initiative currently stalled; this tension reflects outputs, vision and planning in Figure 1, and 'thought experimentation' in Figure 2b, including the move from species to ecosystem management.

Acquiring breadth and depth of knowledge Uses deeper knowledge (detail) Uses broader (transdisciplinary) knowledge
River management Detailed understanding of hydrology/hydraulics in rivers allowed detailed understanding to feed into environmental flow requirements, allowing dam releases and timings to cover bottom loops and input-process-output.
Flow knowledge meshed with geomorphic, biotic and increasingly also with social needs/ecosystem services from these rivers.Joint (transdisciplinary) vision setting in catchments is a feature of catchment management, addressing loop 6 in Figure 2a and cascading downwards from there.Broad knowledge of river systems provides sound basis for 'thought experiments' as detailed in Figure 2b, resulting in shifts in objectives.This breadth of transdisciplinary knowledge allowed for a complete AM cycle (Figure 1).

Learning and unlearning Involves mainly cycles of (closely related) iterative learning Uncovers need for big changes, demands unlearning
Elephant management Rapid learning during refining elephant culling in the 1970s and '80s illustrates feedbacks 1 and 2 of modern-day AM protocol (Figure 2a); represented input-process-output cycle.Recently, a battery of TPCs were introduced of which only one (the large tree TPC) was shown to be exceeded, possibly owing to elephant activity.This has extended the cycle in Figure 1 to include outcomes.
In spite, or perhaps because of, capacity to cull efficiently, the big change in thinking about whether and how to control elephant numbers started in the 1990s.Much 'hard-wired dogma' about strict predictable relationship between elephant numbers and biodiversity effects had to be unlearned, particularly as loop 4 in Figure 2a was addressed.This unlearning allows a move towards a full AM cycle (Figure 1) for elephant management, assisted by 'thought experiments' (Figure 2b).Even so, there may still be some unlearning constraints required when compared to AM cycles in fire or rivers.

Applying single-, double-and triple-loop learning
Uses mainly single-loop (and some double-loop) learning Uses mainly triple-loop (and some double-loop) learning

Artificial water provision
Adjustment of surface water provisioning was fine-tuned based on perceived optimal distribution of herbivores before 1990.Recent guidelines have emerged around the desirability of particular artificial waterholes or dams being present and several significant research projects have assessed the impacts around waterholes.This supports bottom loops and process-outputs phases.
Heterogeneity paradigm (eds.Du Toit, Rogers & Biggs 2003) internalised in 1990s, associated with reflection.Waterhole deployment results from previous decades had led to triple-loop learning, and questioning of previous stable equilibrium notion.Led to widespread closing of artificial surface water.Coarse-scale reflection led to thought experiments, and partly lagged implementation.

Learning though patient reflection and rapid responses
Learning mainly through rapid responses Learning mainly through patient reflection TPC exceedances TPC protocols embedded in KNP management since 1999 produce an important assessment.Sequence of events in which TPCs were exceeded in each theme (e.g.fire, water, alien invasions) has led to some learning based on rapid ongoing responses, involving mainly lower loops and planning to output phases of Figure 1.
Although TPCs were a key catalyst, it was the patient reflection associated with the exceedance events which augmented overall learning.'Mid-term assessment' in 2000 [1] included a light element of reflection; some more recent initiatives listed have better reflection qualities.Where more reflection took place (e.g.fire and river management) TPCs were coupled to broader mechanisms and used most loops in Figure 2a.

Focusing on individual and extended group learning
Learning can often be at individual level (though focused groups can be involved)

Most learning achieved by interaction in heterogeneous groups
Invasive aliens Rules for monitoring can be tested by an individual or fairly homogenous group, for example, alien invasive plant thresholds and responses.The latter has multiple scales of operation, which embody policy alteration at these scales (i.e.double-loop learning).

TABLE 4:
The relationship between assessment and reflection for each paired case in Table 3, with comments on overall learning and the relationship to adaptive management.

Duality and example used Relation between assessment and reflection Comments regarding overall learning Link to adaptive management 1. Prior knowledge vs. new experimentation
Fire management Reflection resulted partly from prior beliefs of experts (which also defined certain contexts, such as the original belief in the ability to actually influence area burnt), and partly from results and experience derived from assessments.Assessment patterns were clearly also influenced by prior reflection.Assessment helped track operational management, with current management exceeding one (variation in intensity) of the two current TPCs (the other one, no longer exceeded, is patchy pattern).
Commonly held to have been vibrant.Major knowledge gap currently being researched is relationship between pyro a -and biodiversity.Such major questions appear to be partly influenced by results, but more by current popularity of beliefs, thus at least denoting responsiveness.May be important that fire is perceived as mainly an internal issue (controllable by park, with exception of some boundary issues).
Several generations of learning within 15 years: first a conceptually driven change to patches and then an experience-and assessment-driven change to allow multiple ignition sources; patch paradigm currently being challenged experimentally, also links between pyro a -and biodiversity.Within paradigms, monitoring has informed management, which struggles to address the intensity threshold exceedance.A full AM cycle appears to be operational.

Threatened species management
Assessment-type activities aimed at keeping target species monitored and populations growing were largely separated from reflection.Sheer size of task of incorporating all threatened species acted as impetus for reflection on how prioritisation of target species might occur for assessment, also in terms of inevitable change, and thus acceptance of changing contexts.The range of specific and sometimes competing values in this regard is not yet aligned within the organisation.
Both single-or double-loop learning (related to particular species) and double-or tripleloop learning occurred, the latter only far later (and currently stalled).
Adaptive processes used in single species conservation, to some effect.More abstract overall framework has been used for prioritisation in KNP, but was not successful in practice.This tension needs further resolution at higher levels in AM cycle.There is currently no champion driving this.

Exceeding TPCs and related reflections
Originally reflections gave rise to TPCs.TPC usage results in questions being raised (also in this paper) about whether the tool is being used too mechanistically, suggesting a disconnection between assessment and reflection.Best to regard TPCs as catalytic to a more productive reflective processes, but also through the necessary medium of shorter-term responsiveness?
Although some learning is taking place, a way of using TPCs that enables more learning should be sought.Understanding decision making systems would be useful, as is routine use of system diagrams underlying TPC mechanisms.The current development of processes to expose various scientific and experientially based management mental models is assisting in learning (see also Gaylard & Ferreira 2011).
TPCs materially assist in leveraging action.Individual TPCs are healthily challenged.Varied critiques should be encouraged to encourage reflection at levels broader than just the use of individual thresholds.Need to keep watch for decoupling of the lower level (loops 1-3) and upper feedbacks (4-6) in Figure 2a.participants regarded this as an improvement compared to previous such initiatives.There was fair buy-in for the overall goal to plan and manage actively for river-related objectives and the majority of interviewees appreciated that they had been involved actively.They understood their roles in the process and, moreover, that their behaviour had changed to include more concern for and action on river issues.Several mentioned other administrative systems as competing for valuable time when it came to executing the river objectives.
The results of the study suggest that rangers participated in a nested way in assessments, were partly involved in reflection, and subsequently learnt in a meaningful way.
Roux, Murray and Van Wyk ( 2009) interviewed eight SANParks scientists of KNP Scientific Services to examine the extent to which their department could be considered a learning organisation.Interviewees considered 11 learning principles derived from literature and regarded statements reflecting application of nine of these as being 'largely true'.
Scores reflecting organisational capability and information and data processing were far lower, suggesting caveats in understanding and/or implementation, perhaps relative to the growth in other learning areas which placed higher demands on these.With reference to the institutional model of Anderies, Janssen and Ostrom (2004), Roux et al. (2009:49) found that although the internal learning system (in this specific context referring mainly to within Scientific Services in the KNP) works fairly well, the greater encompassing learning systems (e.g.catchment management agencies) do not necessarily do so.Links from the subsystem to these bigger systems may also have been unclear: 'Good knowledge workers [within KNP] must step outside of their usual domain to engage issues within the larger learning system and to deal with complexity and response scenarios across a range of organisations and scales' (Roux et al. 2009:49).However, organisational context and ethics sometimes complicate moving beyond the usual domain and functioning effectively, even for those who are willing and keen to do so.This study targeted learning, and although reflection was explicitly probed and found to be taking place to some extent within the organisation, the report did not deal directly with assessment and inter-relatedness as discrete or traceable topics.Self-evaluation within the mentioned project suggested limitations of such mechanistic analyses.This led to a more nuanced articulation in which each action was seen as a paradox, of which facets of both components needed simultaneous consideration, similar to the seven paired strategy options described by Roux et al. (2008) and which we describe here as learning dualities.They include: • considering both prior knowledge and new evidence • practicing conceptual as well as practical learning • acquiring breadth and depth of knowledge • learning and unlearning • applying single-loop, double-loop and triple-loop learning • learning though patient reflection and rapid responses • focussing on individual and extended group learning.
The maturity, purpose and operating realities of an organisation will cause the balance between these dual needs to change over time.Without this tension, it is unlikely that adaptive learning can take place effectively and, as such, adaptive management itself should be expected to lead to conscious or subconscious shifts in this balance.Roux et al. (2008) found that these paired dualities, if discussed in depth amongst stakeholders to determine how effective they were at practicing adaptive learning, were extremely useful in eliciting improved understanding of the learning process.Using these dualities as a framework against which assessment and reflection (and the relationship between them) in the studies mentioned in Table 1 can be compared should help us to gain knowledge of what was learnt, and how this learning relates to adaptive management.

Assessment, reflection and the learning dualities
We propose that the opposite endpoints of each duality appear to represent an emphasis towards either assessment or reflection (Table 3) as defined in this paper.We have intentionally selected examples (from sources in Table 1) to illustrate this from a wide range of themes.We use one theme for each duality so that assessment and reflection for the specific duality can be juxtaposed.Although assessment is commonly thought of as a precondition for reflection, we try to examine the relationship between the two critically in the narration in Table 4.We also comment on how the authors gauged overall learning to have proceeded and the inferred links to adaptive management.This helps cover the full organisational perspective of the cycles represented in Figures 1 and 2, including assessment, reflection and TABLE 4 (Continues...): The relationship between assessment and reflection for each paired case in Table 3, with comments on overall learning and the relationship to adaptive management.

Duality and example used
Relation between assessment and reflection Comments regarding overall learning Link to adaptive management

Individual-focused vs. larger heterogeneous group-based learning
Alien species invasion Assessment related mainly to immediate threats at different scales or phases of invasion, and relatively isolated from reflection.This is now changing, perhaps resulting in a new general paradigm.
Boost in learning took place when invasive organism and emergent disease ideas were compared.Wide stakeholder base (e.g.AHEAD) provides a very wide and fertile basis for idea sharing.
Adaptive responses geared mainly at 'acceptable' disease impacts, taking multiple land use lobbies into account.Acceptance of surprise underscored by scenarios in AHEAD.
Heterogeneous inputs to such processes appear to raise chances of success.
a , pyrodiversity refers to variation in fire pattern, seasonality, intensity, etc. AHEAD-GLTFCA, Animal and Human Health for the Environment -initiative for the Greater Limpopo Transfrontier Conservation Area (also see http://www.wcs-ahead.org);AM, adaptive management; KNP, Kruger National Park; TPC, threshold of potential concern (Biggs, H.C. & Rogers, K.M., 2003, 'An adaptive system to link science, monitoring and management in practice', in J. du Toit, K.M. Rogers and H.C. Biggs (eds.),The Kruger Experience: Ecology and management of savanna heterogeneity, pp.59−80, Island Press, Covelo; see also example [2] in Table 1).
integration, and their influence on and from decision making.
In this way we hope to illustrate whether, through decision making, learning leads to improved action, and action to further learning, as described by Kolb (1984).
From Tables 3 and 4 it becomes clear that cycles of both assessment and reflection are asserted to have led to learning, albeit at different scales and paces and with varying time frames for decision making and management implementation.Effective adaptive management, however, does require a timely response to learning.This has not always been the case in the KNP examples, and assessment, reflection, and response times have often been decoupled.This led to reduced effectiveness, which is probably related to either lagged learning or lagged responses to learning.However, not all lags are undesirable; in fact, they can allow for adequate reflection (Cilliers 2006).

Discussion of the learning duality approach as applied to the case studies
The examples illustrating the learning dualities highlight circumstances which emphasised assessment or reflection, how they appeared to inter-relate, and allude to some obvious gaps (Tables 3 and 4).This strongly suggests that it is meaningful to distinguish between assessment and reflection, because they appear to have different roles and can lead to different outcomes and are apparently often decoupled.
Inter-relatedness of feedbacks appears to augment and/or be the result of deeper reflection, although such evidence is limited at present.In addition, the intuitive impression that assessment precedes reflection seems not to be well supported by these narratives.We believe that in order to span the various needs of the complex challenges in natural resource management, both assessment and reflection are needed, perhaps concurrently.Intuitively, we seem to want them to be complementary or even synergistic, although our case material suggests they do not often tend to feature'naturally' in the same initiative.The relationship between them is often complex (Tables 3 and 4) and it may require explicit effort to create any such synergy or complementarity.It seems that assessment is often linked to acting on the answers that emerge, whilst reflection may generate more depth by asking more questions.The challenges we face may need both action and concurrent contemplation if we are to stay both responsive and resilient in future.Is it possible that reflection is meant more to consider our actions and interventions, whilst assessment is meant to trigger more immediate interventions?
Assessment and reflection can be considered to constitute an overarching duality required for effective learning and, where they lead to decision making, also then for adaptive management.Kolb's (1984) learning cycle refers to observations and reflections following from experiences, with reflections then being distilled into abstract concepts for re-testing in reality.He details different learning styles, of which a rework by Honey and Mumford (as cited in Businessballs [n.d.]), describes activists, reflectors, theorists and pragmatists.We suggest that assessment may be something carried out more by activists and pragmatists, whilst reflection is an activity more related to reflectors and theorists.Importantly, these four styles together constitute a full learning cycle of Kolb (1984), possibly implying that what we interpret as a duality between assessment and reflection could also (and perhaps better) be expressed as assessment and reflection complementing each other in a dovetailed way.This might also explain why we interpret the preliminary data (Table 2) as suggesting a disjunct between assessment and reflection.One likely explanation is that if they are not executed together in the right dovetailed way -that is, if there is an imbalance between them (as there often seems to be in our case studies) -learning is in some sense incomplete.We accept that under the circumstances that unfold in the situations we normally deal with there may be a need for later studies to try to re-align the correct balance between them, and hope that this paper may have contributed in this way.Conversely, we suggest that if one can design complementary use of assessment and reflection appropriately from the outset, learning may be more effective.
As with the learning dualities, or in the same way that learning styles (Kolb 1984) are seen as mutually exclusive at one instant, there is a necessary tension between assessment and reflection (Roux et al. 2009).Presumably, shifts in emphasis between them over time are, or should be, adaptive.A generalised schematic of the inter-relationship of key elements of assessment and reflection, supported by our analysis, is presented in Figure 3. Whilst this represents the existing tensions, striving for an improved relationship should also include issues of scaling.With some exceptions, reflection conventionally tends to involve coarse-scale issues and assessment finer-scale issues, as seen in our examples.Challenging this 'convention of scale' by also reflecting at finer scales and assessing at coarser scales, and interlinking assessment and reflection across these scales, could lead to more useful feedbacks.
If better understood and linked, a well-conceptualised and tailored combination of assessment and reflection (i.e. for the particular situation at a particular time) may support enhanced purposeful learning.Perhaps we have not made the differences between them sufficiently explicit to be able to exploit the opportunities of this fuller learning.This paper illustrates that they do not have a fixed, linear or unidirectional relationship.Even when used independently, but particularly when used together, they appear to underpin learning and, when they result in decision making, also adaptive management.Emphasis on one or the other leads to different types of learning (duality 5, Table 3) and evoke different adaptive management feedback loops (highlighting loops 1-3 and 5 for assessment, and loops 4, 6 and 7 for reflection; Figure 2a).So, there appears to be a need for both to complete the adaptive management cycle, similar to how different learning styles produce a full Kolb (1984) learning cycle.Our impression is that assessment tends to provide, broadly, the basis for the inputs through output stages, and reflection the basis for the outcome through inputs stages (Figure 1).This may be a consequence of managerial and scientific groups in the KNP being more involved in the implementation and planning phases, respectively, although older staff recall an era (of the so-called 'House 31' meetings and the first adaptively planned management plan in the late 1990s) when managers and scientists appeared to share assessment and reflection more equitably than after the staff restructuring of 2001.Our challenge appears to be, for each case, to couple assessment and reflection more productively by ensuring enough of each (there is often too little reflection) at scales appropriate to the case, and ensuring that they complement each other in the most useful way.
The outcome stage (Figure 1) is a particularly critical step at which considerable reflection is possible.This leads to consequences for implementation, which may, through revisioning, influence planning in the next iteration.Examples where the feedback loop is thought to have been completed, such as rivers and fire management in the KNP, share a key bridge that allows for learning from doing.Reflection at implementation and output stages, shown in elephant management and TPC examples, is also helpful but more to refine technical aspects of management.In addition, thought experiments (depicted in Figure 2b) and modelling can help identify and improve options and enhance the possibility of successfully learning from anticipated doing.In particular, thought experiments that involve both scientific and managerial stakeholders and include elements of both planning and implementation, may assist in coupling assessment and reflection at multiple stages and scales of the cycle.This should lead to more purposeful reflection and learning.
Decoupling of operational and conceptual issues highlights a known tension in natural resource management.This is illustrated in the threatened species example (Tables 3 and  4), where specific knowledge and a lobby for a particular aspect, in this case an individual species, oppose the need to conceptualise ecological frameworks into new management paradigms, which in this case refers to management for overall ecosystem heterogeneity.Assessment at the level of managing an individual iconic species was translated into objectives that were not necessarily in line with the overall park vision.No persistent holistic reflection emanated from either the outcomes phase or the emerging changes in context such as climate change.The reflection that did take place, on the other hand, resulted in an abstract conceptualisation of heterogeneity, which was unacceptable or disconnected from on-the-ground-realities in that it appeared to undermine preservation goals for charasmatic species.
The examples show that when multidisciplinary, multiinstitutional teams come together to reflect in the outcomes stage (in the fire and rivers management programmes and also to some extent in the elephant and alien species invasion programmes) and fully connect the adaptive management cycle though decision making, both learning and adaptive management are deemed more successful.However, these should be accompanied by sufficient assessment.Such perceived success is partly lacking in the current relatively inert threatened species programme and the 'stop-go' artificial water provision programmes, although impressive individual initiatives may have contributed a great deal (e.g.SANParks 2010).Adaptive learning seems to have been deeper and easier when a clear and jointly agreed goal was set, without other conflicting goals, which enabled decision making and implementation.This has not been the case for the threatened species programme in the KNP.Neither was it the case for the artificial water provisioning programme until fairly recently, when a re-evaluation of context and principles (including acknowledged overriding importance of the heterogeneity paradigm and of climate change) led to consensus for implementation (SANParks 2010).

Is the Kruger National Park good at learning in adaptive ecosystem management?
Whilst it is encouraging to note that many narratives in this paper suggest this to be the case, there appear to be major gaps and opportunities.The evidence presented suggests that assessment-type learning predominates and is inadequately linked to reflection.Assessment appears amenable to, and can even be effectively driven by, codification, which enables some stability for decision making.This may be appropriate under conditions of relatively unchanging, or assumed unchanging, contexts, but could result in 'blind-siding' and a 'dulling of the senses' when confronted with changing and newly emerging contexts.Reflection, on the other hand, is often spontaneous or emergent, although, clearly, some structured processes can also be enabling and can lead to more fundamental changes in decision making.The KNP may be guilty of often falling back into a narrowly defined version of adaptive management, which may tend to assume a given context.decades before a change in thinking was accepted, all the while showing welcome technical improvements and refinements within the particular culling context, which proved unsustainable, as 'the only answer'.On the other hand, there is generally also an urgent need for effective assessment processes, as depicted on the left side of Figure 3.
The way forward appears to be, as the dualities suggest, that assessment and reflection should coexist with their complementary function playing out at various scales and with varying strengths.Recognition of dynamic complexity, linked to a system that supports policy-oriented decisions, seems crucial.Such interlinking requires diverse teams working together on problems holistically.For example, in SANParks this could include, in a stereotypic or conventional interpretation, (1) managers who often tend to focus on finerscale implementation and assessment issues, (2) scientists who tend to have a broad, theoretical view, focus on planning and outcomes and, intuitively, like to reflect, and (3) 'bridging' regional ecologists who have an understanding of both planning and implementation components.It has been stressed before that more shared functionality across these cadres results in better integration.The prescription should therefore also include stretching, within the reasonable capability of people, these stereotypes outside of their immediate operational comfort zones and promoting engagement by at least some individual team members across all aspects of the adaptive management process as illustrated in Figure 1.This, in itself, will lead to enhanced achievement of the original dualities or paired options (Roux et al. 2008).
Given the paucity of quantitative empirical evidence (Cook, Hockings & Carter 2009), is it the reality of complex systems that we should expect a mix of fact-based and expert-based evidence and changing ideology to drive management, rather than imagining the trajectory to develop solely on hard factual evidence?How should scientists respond to this?This paper describes that, in reality, several different classes of feedback exist, of which only some are based on hard evidence.This suggests that a mix of these change drivers is inevitable (even if hard evidence plays an increasing role) and that reflection at different scales may help improve decision making and learning amidst this uncertainty, with changing ideology often (but not exclusively) manifesting itself through policy option choice.
Our case histories showed some examples of assessment patterns being driven by decisions made subsequent to reflection, often after a lag period, which may not always be undesirable (Cilliers 2006).These reflections appear, in turn, to have been driven mainly by changing paradigms.
The changing paradigm itself is a consequence of primarily, but not entirely, scientists' reflection and these changed paradigms prompt further management reflection.We have also described how assessment contributes to further reflection, but individually these effects seem to reach tripleloop standing only occasionally (Roux et al. 2009).Thus, a combination of altered expert opinion (including that of managers) and changing values, in some cases together with the appearance of a certain amount of new evidence, appears to lead to change.All three these drivers may co-evolve or be brought together in a more conscious process.Is the overlap of striving for the same goal, although perhaps for different reasons, a key element of practical progress in many cases?
A way forward may be to target several key questions: • What influences the balance between assessment and reflection in particular cases?• Is assessment more prevalent because it is easier for organisations to implement and/or codify?• What is the trade-off between tedious, detailed assessment and a more practical intervention that is not overly simplistic?• Should all systems strive to include and promote personal thoughtfulness and vibrancy, along with other strategies, to prevent becoming rote?• How can we promote more reflection and its meaningful relationship with assessment?
These may be addressed in an enlightened way by using one or multiple approaches to evaluate our success at adaptation and learning, as discussed by several others (e.g.Michael 1995;Narrative Lab 2009;Paehlke & Torgerson 1990;Stankey et al. 2005;Walker & Salt 2006;Weick & Sutcliffe 2001), and incorporating these into wider fundamental learning theory (Kolb 1984).Such initiatives could even re-use the case data mentioned in Table 1.Although they have much in common, each of these approaches also has a unique slant.We propose that sufficient reflection on one or more of these approaches (perhaps emphasising their commonalities) and not only the learning dualities on which this paper has focused, may yield startling and useful consolidations in our particular context of natural resource management.

Conclusion
We have shown that both mindful assessment and reflection are necessary for purposeful learning and adaptive management, and that there is much scope for improvement throughout this interacting system.Consideration to both learning theory and the correct contextualisation of adaptive management helps to determine the dynamic coupling we observe in natural resource management: complex socialecological systems are embedded in extensive dynamic contexts.With this paper we have made a start by looking at empirical evidence available after more than a decade of adaptive management in a park, and relating the patterns to meaningful theory.We hope the review can serve as a springboard for more exact work of this kind and hence further enhance learning.

TABLE 1 :
Studies or initiatives that involved adaptive management in or around the Kruger National Park, rated for their degree of assessment, reflection and interrelatedness of the particular analysis.

Nature of study or initiative Purpose and/or outcome of the study or initiative Rating a Reference Assessment
W. & Biggs, H.C., 2011, 'A critical assessment of adaptive ecosystem management in a large savanna protected area in South Africa', Biological Conservation 144, 1179-1187.SAM presented in four steps; three case histories presented -two from the KNP and one from Australia; progress shown as incremental; more advanced cases allow for institutional reflection; AM should be applied in urgent and contentious cases despite long time needed for maturation 2 1 2 Kingsford, R.T., Biggs, H.C. & Pollard, S.R., 2011, 'Strategic Adaptive Management in freshwater protected areas and their rivers', Biological Conservation 144, 1194-1203.Total score (out of a total possible 30) 20 15 15

TABLE 2 :
Spearman's rank correlations between ten ordered individual scores for assessment, reflection and inter-relatedness in Table1.
p, p-value; r s , r-value.

Table 4
continues on the next page → General schematic of the existing relationships between learning, adaptive management, context specificity and the duality between assessment and reflection, as evident from analyses in this paper and learning theory in terms of the Honey and Mumford modification of the Kolb learning cycle concept (Businessballs n.d.).
The KNP may even have become trapped in a fixed context, which has become inappropriate over time.This is akin to the way in which attitudes around elephant management (eds.Scholes & Mennel 2008) persisted for FIGURE 3: