Leadership Development and a Case for the Use of Taxonomies

by

Shannon Flumerfelt, PhD.

 

Abstract

    For those working to develop leaders, gaining a clear understanding of the developmental processes occurring is helpful.  Leadership development pursuits can become more authentically based and individually focused when based on pedagogical processes including taxonomy development and use.  Utilizing the power of taxonomies allows one to tie together curriculum, assessment and instruction with the understood process of learning for those pursuing leadership development.

 

 Key Words:  leadership development, taxonomies, learning process  

Background

Powerful learning experiences in the pursuit of leadership development, whether garnered through formal programs or informal interactions, quickly lose their potency if the learners experience disconnections between theories and the practical world.  Uncovering a new leadership concept, one that expands the knowledge base, can provide the thrill of discovery.  However, euphoria can turn into disappointment and frustration if the learner cannot translate that knowledge into practice.  Likewise, leadership practice without a solid base of knowledge or adequate values development can prove a shallow experience.  From a different perspective, exercising intuitive leanings as a leader without knowing the “what” or “why” behind the “how” will most likely cause a crisis of confidence at some point.  Leading with strong values is an important leadership practice as well, but believing in something without being able to articulate the theory or research behind the value rings empty.  Values need both knowledge and application, and vice versa, for enrichment and meaningfulness.

Molenda(2002)describes developmental theories of the past, which were typically based on an “either-or” state of affairs.  Either there was growth occurring or there was not.  For instance, Skinner’s operant conditioning theories of behaviorism required a set of stimuli, reinforcers and responses.  These forced choice options for states of development eventually developed into cognitive theories.  This type of development as conceived by Gagne, combined behaviorism with an interpretation of learning theory.  This thinking, however, was still based on the concept of stable states of development over periods of time.  Eventually, modern theorists looked at development in domain-specific modules of development following a distinct timetable and ending in a high level of skill.  This is representative of constructivism.  This viewpoint saw the environment more holistically and considered it complex.  This complexity, in turn, made learning more intricate as well.  Learning was seen as a self-constructed internalized process.  Now domain specific elements are being linked with global elements as identified by contemporary researchers.  Global elements include the architecture of the developing cognitive system, elements such as processing speed, working memory and executive functioning (Kail, 2004). 

This progression in thinking about developmental theories from linear models to systems thinking is why a case for the use of taxonomies for leadership can be made at this time in history.  Namely, taxonomies and development systems are compatible.  Taxonomies match systems thinking in that they allow the learner and/or facilitator to examine the process of learning and understand how developmental steps relate to each other.  Taxonomy use creates a better understanding of exact learning processes, the leader’s scope and sequence of learning, and the relationship of knowledge to values to application, and any combinations of those relationships. 

Taxonomies, if properly used for leadership development processes, can also be used for evaluation and assessment.  The properties of taxonomies exceed other approaches to evaluation in that the information generated from a taxonomy assessment tool is more systemic.  Taxonomy-based assessment data can uncover where relationships between learning points, sequences, relationships, and hierarchies of processes occur.  Assessment tools based on more linear models, such as checklists or rubrics, do not reveal systemic relationships.  If leadership development is understood as a process, then taxonomy development and use can help in understanding that process and in assessing the progress of development occurring.  The time is appropriate to elevate leadership development initiative design and leadership assessment to the realm of taxonomies.

Taxonomy Definitions

 If leadership is conceptualized as a system or taxonomy of knowledge, values, and application development, then understanding the growth of the emerging leader is much easier to do.  Taxonomy is derived from the Greek word “taxis,” meaning arrangement of division and “nomos” meaning law.  Bruno and Richmond (2003) state,

 “taxonomy is the science of classification according to a pre-determined system, with the resulting catalog used to provide a conceptual framework for discussion, analysis, or information retrieval.  In theory, the development of a good taxonomy takes into account the importance of separating elements of a group (“taxon”) into subgroups (“taxa”) that are mutually exclusive, unambiguous, and taken together, include all possibilities.  In practice, a good taxonomy should be simple, easy to remember and easy to use.”  (p.45)

 

They provide further definition by explaining that taxonomies are,

“. . . structures that provide a way of classifying things—living organisms, products, books—into a series of hierarchical groups to make them easier to identify, study or locate.  Taxonomies consist of two parts—structures and applications.  Structures consist of the categories (or terms) themselves and the relationships that link them together.  Applications are the navigation tools available to help users find information.”  (ibid, p.45)

Taxonomies have two uses, as explicit descriptions of results desired and as customized instruments by which to measure those results in a holistic way.  As a description of results desired, a taxonomy accounts for behaviors desired.  As a systemic measure of development, taxonomies provide a picture of both what learning has occurred and how it has occurred.  Taxonomies provide a way for identification, discovery and delivery of information about leadership development processes.

Taxonomy Development

If taxonomies are useful in gaining a holistic perspective on leadership learning, then how are they developed?  How does one decide what descriptors of behavior should be included in the taxonomy?   There are three typical approaches to taxonomy development. The first is systematic observation, which requires looking for contrasting styles and behaviors.  A second approach uses ethnographic or quasi-ethnographic observation to form descriptive categories of styles and behaviors used.  The third turns to discourse analysis which uses verbal utterances of participants based on issues such as linguistic structure, personal reference, form, function (Fulford, Sakaguchi, 2002). 

Basically, the information for the taxonomy is derived from the learner, through one or more of these three methods of observation, ethnographics or verbal utterances.  Taxonomy development methodology can use all three methods or only one.  These three methods require focusing on the learner’s behavior, characteristics and theoretical research, or words and body language.  In developing a taxonomy, a “learner-centered” perspective is gained.

Once the taxonomy is developed, an assessment tool to accompany it is developed.  The taxonomy and assessment tool are then piloted.  They are tested and eventually evaluated for reliability and validity, if so desired.  The development of a taxonomy can be broken down into procedures designed to match desired objectives. 

For instance, Bangert-Downs and Pyke (2002) describe a single procedure they used for taxonomy development:

 

1.      Develop taxonomy for testing

2.      Train researchers, test taxonomy

      3.   Analyze data obtained from field test on taxonomy items

 

This methodology does not allow for generalizations, and does not develop the taxonomy well for consistency.  However, for use in the localized, specific setting for which it was developed with a small number of participants, this procedure is adequate for gaining a better understanding of learning processes for that circumstance. 

A second example of taxonomy development is found in the work of Fulford and Sakaguchi (2002).  They used three procedures in order to accomplish the two goals of developing a taxonomy and checking for ease of its use.  The first procedure is outlined:

1.  Put together a taxonomy draft based on theoretical research

2.  Provide preliminary definitions of utterances based on theoretical research. 

The second procedure involved:

1.      Ask experts to rework taxonomy and definitions

2.      Redo taxonomy and definitions based on experts’ feedback

3.      Develop assessment tool based on taxonomy and definitions

4.      Collect data (videotapes, surveys, etc.)

5.      Train experts on use of assessment tool, use interactive approach to guarantee
                   consistency

6.      Develop Answer Key from experts’ session for assessment tool

7.      Have independent, non-expert, raters evaluate data using assessment tool

8.      Run kappa test of significance to determine evaluator reliability (testing experts to raters
                   consistency, Z-scores)

9.      Get informal feedback on assessment tool’s training, instructions, definitions, ease of use

The third procedure involved:

                    1.  Looking for agreement by running statistical tests on individual components of
                                     the taxonomy

 

The methodology in this second example aims at reliability and validity, and therefore, has potential for more generalized use than the first example did.  It also includes the input of experts in the early development stages to ensure content validity, along with statistical tests.

A third example of taxonomy development is presented by Moss, Valenzi and Taggart (2003).  They had three goals, including constructing a clear taxonomy of behaviors, developing an instrument to measure the constructs of the behaviors in the taxonomy, and, finally, demonstrating that actual behaviors match the processes and behaviors in the taxonomy.  They used several procedures, beginning with:

  1.       Item generation and taxonomy development using standard scale development

A.     Determine what to measure:  constructs, definitions

B.     Generate item pool:  large list of items

C.    Review of item pool by 3 experts: smaller list of items

D.    Sort lists by categories

E.     Reduce item list

The second procedure required:

1.  Exploratory factor analysis for item reduction, reliability assessment

A. Determination of format for measurement

B.  Two sample administration

The third procedure involved:

1.      Use of LISREL factor analysis

2.      Fit estimate

3.      Internal consistency by construct reliability, coefficient alpha, average variance extracted

4.      Discriminant validity by Phi estimates, chi-square differences

A fourth procedure utilized:

1.      Construct validity assessment through various measures

A final procedure concerned:

1.      Test for actuality of behaviors

This approach, while more complex, does confirm the taxonomy design matches with actual behaviors and holds potential for use in different settings. It has the same strengths as the second approach, but takes the development one step further to ensure the taxonomy, already developed and tested, matches reality.

In general, there are four goals that can be accomplished in taxonomy development:

1.      Develop a taxonomy

2.      Develop an assessment tool

3.      Check to be sure taxonomy relates to actual behaviors

4.      Correlate taxonomy to other variables

Depending on the goals desired, research methods should be selected as the examples above.  The choices for the development stages can use an action research, qualitative or mixed methods (qualitative and quantitative) approach.  The approach selected, as indicated by the three examples provided above, should match the desired goals and will determine the ultimate use of the taxonomy. 

How to Use Taxonomies in Leadership Development Programs

Taxonomy development and use can be very helpful in understanding the effectiveness of various leadership curriculum; the intensity, pace, scope and sequence of how leadership is learned; and the relationship of different areas of learning to each other.  Two examples of how taxonomies have been used in leadership development settings are described.

Example 1.  A select group of high school students participated for seven months in a leadership development academy at a Midwestern high school.  After operating this academy for several years and gathering data through observations and verbal utterances, it was apparent to the facilitators that leadership growth was occurring. At the academy students kept journals, took a variety of self-assessment inventories and participated in pre-academy and post-academy interviews. 

Through this student-generated data, and through inventories completed by parents and teachers, observations of counselors and facilitators, a list of descriptors was developed that delineated how the students behaved when leadership growth had occurred.  By working with this data and conducting a formal qualitative study, patterns of data were arranged into construct areas.  These constructs were then used to explore emerging themes in the data.   Relationships between and among these constructs were designed into a taxonomy.

Since the group of students in the formal study was small, totaling twelve, this taxonomy cannot be used in other settings, until its assessment tool has been tested for reliability and validity.  Its usefulness, however, is in helping the facilitators of the leadership academy understand how learning was occurring for these adolescents.

What was learned is that growth was occurring in three areas, classified by content-related, developmental and accelerator issues.  Content-related issues included ideas such as excitement about learning about the existence of leadership tenets, concern for how to become an impactful leader and struggling with how to lead one’s peers. Developmental issues included items such as assessment of one’s own state of leadership, questions about how to discuss concerns in the academy group, and worries about whether parents and friends would understand what the students were thinking about. The developmental and content-related concepts were arranged on a grid of phases and levels in a matrix. This was done by marking the section on the grid where the students appeared to be constructing meaning.  After completing the grid, there were some coded, but unsorted, data that lead to the identification of a third conceptual issue of accelerators, consisting of four possible components.

Sequencing of learning was indicated as well on the matrix.  It became apparent that the students might be learning simultaneously in more than one section of the matrix.  The learning could be occurring in multi-level and/or multi-phases fashion simultaneously.  The idea of “multi-tasking” on different phases and levels within the same period of time, and even recycling through phases and levels, emerged.  The codifying and sorting by conceptual issues on the matrix was helpful in understanding the process of development.

Themes were developed next and included spiraling, inter-connected areas of growth on three levels and three phases; the impact of four accelerators based on pace and intensity of growth, and a system of leadership development growth by combining the levels, phases, and accelerators.

The conclusions from the development of this taxonomy were drawn from the findings of the students’ growth as individuals and as a group as well as from the discoveries obtained by combing through the students’ profiles and searching for patterns of development.  The most obvious conclusion drawn from the taxonomy was that the students did in fact experience development.  Gathering an understanding of their learning processes provided a conceptual framework by clarifying how positive development occurred.  The additional conclusion was drawn was that there was a system of development that occurred, organized into a taxonomy of leadership development (Flumerfelt, 2000). 

Example 2.  Adult students enrolled in a graduate degree program for school administrator preparation at a Midwestern university engage in leadership development. After operating the graduate degree program for several years, it became obvious to the faculty that student leadership growth was occurring.  Student data had been gathered through student work samples and program activities, student testimonials, student interviews, and post-graduate surveys.  This degree program is founded on the Interstate Leaders Licensure Consortium (ISLLC) standards (1996). These standards are arranged in a taxonomy, based on knowledge, dispositions and performances of eight competency areas. Each competency has a list of behaviors for knowledge, dispositions and performances.  

In this example, since the categories were developed for the taxonomy of administrator leadership development represented by the ISLLC standards, an assessment tool was developed to measure student development of the standards.  The assessment tool, in the early stages of design, is being tested by faculty for use in examining learning processes.  In its piloting stages, the assessment tool is providing faculty with ethnographic data about leadership development.  Students who are pre-service administrators report (through student work samples) a different sequence of learning than practicing administrators do. Pre-service students tend to lead their learning with knowledge acquisition or dispositions first, then performance last.  Practicing administrators tend to lead with performance, followed by knowledge and values. 

The data from this taxonomy and assessment tool have shown that students are experiencing different learning processes based on ethnographic differences.  This means that differentiated instructional approaches in courses may be needed or that students might need to be separated in programs based on administrative experience to match sequences of learning (Flumerfelt, Ingram, 2005).

Implications

The benefit of taxonomy development is that it is an engaging process; it holds merit for enhancing the interpersonal relationships between facilitators/faculty and participants/ students.  Obtaining this type of in-depth information about participants’/students’ learning represents alternate forms of assessments.  Alternate assessment is valuable for those who wish to work on a more personalized and less standardized basis, as it creates an understanding of the individual learner, his/her needs and preferences for learning. Alternative assessment methods tend to be narratives, rather than single item checklists.  Overall, the development of a taxonomy provides a unique opportunity to gain increased understanding about leadership learning.  Furthermore, taxonomies, when developed into good assessment tools, hold promise for changing human resource procedures into more individualized and personally relevant interactions. 

Conclusion

The implications for taxonomy development and use in terms of enhancing perspectives of facilitators and instructors of leadership development programs are significant. The student-centeredness built into the process of taxonomy development, the implications for authentic assessment tenets, and the opportunities for examination of results of leadership development initiatives are all noteworthy benefits.  Taxonomy development and use provides more information than simple checklist evaluative procedures or rubric-based methods.  Taxonomies provide a picture of the way that learning has occurred and provides documentation of the journey of leadership development.

 

References

Bangert-Drowns, R.L. & Pyke, C. 2002.  “Teacher ratings of student engagement with educational software:  an exploratory study.”  Educational Technology Research and Development, 50:  23-38.

 

Bruno, Denise & Richmond, Heather.  2003.  “The truth about taxonomies.”  Information Management Journal, 37: 48-50, 52-53.

 

Council of Chief State School Officers, State Education Assessment Center.  1996.  “Standards for school leaders.” Interstate School Leaders Licensure Consortium.

 

Flumerfelt, S.  2000.  “A qualitative case study of adolescent leadership development.”  Dissertation.

 

Flumerfelt, S. & Ingram, I.  2005.  “An examination of student leadership plans as they relate to leadership standards and intended practice.”  Unpublished manuscript.

 

Fulford, C. P. & Sakaguchi, Greg.  2001.  “Developing a taxonomy of interaction strategies for two-way interactive distance education television.” International Journal of Instructional Media, 28:375-396.

 

Fulford, C. P. & Sakaguchi, Greg.  2002.  “Validating a taxonomy of interaction strategies for two-way interactive distance education television.” International Journal of Instructional Media, 29:79-92.

 

 Kail, R. V.  2004.  “Cognitive development includes global and domain-specific processes.”  Merrill-Palmer Quarterly, 50:445-455.

 

Molenda, M. (2002).  A new framework for teaching in the cognitive domain.  Syracuse, NY:  ERIC Clearinghouse on Information and Technology. (ERIC Document No. ED470983)

 

Moss, S.E.; Valenzi, E.R. & Taggart, W.  2003.  “Are you hiding from your boss?  The development of a taxonomy and instrument to assess the feedback management behaviors of good and bad performers.”  Journal of Management, 29:487-510.

 

About the author: