Comprehensive Evaluation of the "Finding Home" Program

**We decided to bold the sections that sum up what we’ve done here — enjoy!**

A Comprehensive Evaluation of the “Finding Home” Program – Year Four

July 5, 2013

Prepared by:
Marissa Jane Ballard, Coyote Trails School of Nature
Mären Burck, Coyote Trails School of Nature
Jon Pozderac, Independent Contractor
Samuel Thatcher, Independent Contractor

ACKNOWLEDGEMENTS

We greatly appreciate the passion and dedication it takes to create and run such an in-depth and ambitious program. We thank Susan Cross for giving us roots; Lynne Reardon for her relentless organizational skills; Joe Kreuzman for pulling, pushing, supporting, and inspiring; Oregon University Extension, Bear Creek Watershed Education Partners, and Bear Creek Watershed Council for working with us and their dedication to spreading knowledge and appreciation; participating teachers for leaping into a brand new program with open minds and getting their own time in the dirt; and our ancestors for passing on the skills we use to inspire, teach, and learn.

Funding for this evaluation and the Finding Home program was provided by the Gray Family Fund of the Oregon Community Foundation. We have reached so many people thanks to the generosity and continued passion for nature education of the Gray Family.

We thank you.

 

TABLE OF CONTENTS

Introduction

Program Description
Program Objectives
Expected Outcomes

Methods
Student Pre-Test and Post-Test Surveys
Appreciation and Attitude Survey
Journaling Prompts
Interviews
Teacher Feedback
Attendance Tracking

Results
Attitude/Appreciation Survey
Knowledge Survey
Selected Responses to Journal Prompts
Gait Patterns
Foot Morphology
Mapping
Wild Edible Plants
Debris Hut
Bow Drill Fire
Contour Line Drawing
Art Project Assessments
Student Interviews

Discussion
Objective 1
Result 1
Objective 2
Result 2
Objective 3
Result 3
Objective 4
Result 4
Objective 5
Result 5
Teacher Outcomes
Objective 1
Result 1
Objective 2
Result 2
Objective 3
Result 3

Conclusion

Bibliography

APPENDIX A Pre-survey
APPENDIX B Pre-survey Directions
APPENDIX C Post-survey
APPENDIX D Post-survey Directions
APPENDIX E Permission and Release Form

List of Tables

1. Attitude/Appreciation Survey Results with a student sample size of 270
2. Number of each answer given by a student for each question
3. Knowledge Survey results with a student sample size of 270
4. Number of each answer given by a student for each question
5. Number of absence and attendance rate per activity
6. Sample of 10 students depicting 100% improvement by students
7. Sample of 10 students depicting 100% improvement by students

List of Figures

1. Graphical representation of Table 1 depicting the pre- and post-test mean averages
2. The average point increase between the pre-and post-survey answers given by the students
3. Pre- and post-knowledge survey results
4. Average difference between pre- and post-survey results by question
5. Animal Powers A
6. Animal Powers B
7. Animal Movement: Gait Patterns A
8. Animal Movement: Gait Patterns B
9. Foot Morphology A
10. Foot Morphology B
11. Mapping: Path
12. The Big Four
13. Observational Skills: The Big Four
14. Debris Hut A
15. Debris Hut B
16. Bow Drill Fire A
17. Bow Drill Fire B
18. Contour Line Drawing A
19. Contour Line Drawing B
20. Pine Needles
21. Cattails
22. Oaks
23. Coyote
24. Cougar
25. Rabbit
26. Map of Home A
27. Map of Home B
28. How to Make a Debris Hut
29. How to use Pine Needles
30. How to Use a Bow Drill Kit
31. Question One Responses
32. Question Two Responses
33. Question Three Responses

INTRODUCTION

In the seasons from January until June 2013, Coyote Trails School of Nature, an independent non-profit based in Southern Oregon, Oregon University Extension, Bear Creek Watershed Education Partners, and Bear Creek Watershed Council collaborated on a multi-season program to bring educational outdoor education to four different Title I schools (50% or more students from low-income families) in the Medford-Phoenix (Oregon) area. This “Finding Home” program was conducted over an extended period to expose students to their local natural landscapes during different seasons. Coyote Trails provided hands-on, inquiry-based experiences with early human technologies, awareness skills, and wildlife tracking as a way to connect students to the natural world and the cycles and interactions within it, as well as increase their appreciation for it. Our time with students allowed them to study different aspects of early human life and connect it to their own, focusing on how human needs relate to the needs of a natural area.

Previous evaluations of the Finding Home program were useful in guiding Coyote Trails instructors in setting goals, determining objectives, and choosing methods for measuring the effectiveness of the program. This year’s program design included administering similar assessments and interviews; however it focused on measuring students’ knowledge of early human uses of natural materials and their understanding of basic ecology in a forested area and riparian zone.

Program Description

The Finding Home program was designed to provide fourth and fifth grade students with an opportunity to have multiple and regular experiences working with hands-on skills, nature-based art projects, and awareness skills. Instruction was provided by environmental educators and a resident artist in the students’ own school yards and surrounding areas at least eight times during the academic year. The program served students in four different Title I schools in the Medford and Phoenix-Talent districts in Jackson County, Oregon.

Staff from Coyote Trails ran the program, visiting the same eight classrooms over the course of five months through the winter and into spring. The program was unable to start at the beginning of the academic year due to limited staff to coordinate, plan, and instruct. This, however, did not affect the number of visits for field experiences of art-based instruction.

The Finding Home program’s learning objectives were established prior to instruction, in alignment with state and federal standards. Instructors assessed student success using program objectives as well as classroom curriculum needs. Standards from multiple content areas were met (art, English, history/social sciences, science, and physical education).

Meeting in the students’ school yards over the course of multiple seasons, and practicing awareness and observation skills, allowed students to observe how an area changes seasonally, grow a deeper appreciation for the area, and learn about the connections humans had to make with the landscape around them in order to survive.

In May, near the culmination of the program, students attended a field trip at the Coyote Trails Nature Center in Medford, Oregon. The students spent time with instructors from partnering organizations, Oregon University Extension, Bear Creek Watershed Education Partners, and Bear Creek Watershed Council, participating in two workshops focused on the Bear Creek and riparian zones. The workshops at the Coyote Trails Nature Center challenged students to interpret a new natural area and find ways to provide for basic human needs using the natural materials found there, considering the interactions between natural living and non-living (e.g., rock, soil, water, dead plant) elements. Students also created nature-based art projects with the intention of using them to teach other students their age. Every student chose one of his or her creations to be displayed at the Coyote Trails Nature Center during the month of May. The public, family, and volunteers were invited to come view the projects.

Program Objectives

The overarching objective of evaluating the Finding Home program was to assess if providing students with multiple contacts with an outdoor area increased their knowledge and understanding of the natural world, its resources, and a human’s place in it, while providing opportunities for students to ask questions and find the answers through their own experiences. The program also intended to assist teachers in aligning outdoor experiences with classroom curriculum and federal and state standards.

The student objectives included the following:

1) Students will increase their understanding of natural systems and the human’s role in it.
2) Students will increase their appreciation for the natural world.

A project‐specific objective for students was:

3) Students will have opportunities to share created work with the larger community.

The objective for the eight teachers whose classes were selected for the project was as follows:
Teachers will increase their ability to align and integrate effective experiential, outdoor-focused, place‐based, and inquiry‐based learning into the curriculum.

Expected Outcomes

Based upon the program objectives, expected outcomes were determined. The expected student outcomes were as follows:

1) 90% of the 8 classes of 4th/5th grade students will take part in all 5 field experiences, 4 classroom visits, and contribute one of each student’s art products to the community event.

2) Student knowledge about the natural system will increase significantly (at the .05 level) from pre- to post-survey.

3) 80% of interviewed students will demonstrate increased knowledge of the natural systems.

4) Student appreciation of the natural systems will increase significantly (at the .05 level) from pre- to post-surveys.

5) 85% of interviewed students will demonstrate increased appreciation for the natural world.

The expected teacher outcomes were as follows:

1) 90% of teachers will report increased comfort with including experiential, outdoor-focused, place‐based, and inquiry‐based learning in their curriculum.

2) 90% of teachers will report increased likelihood to integrate such learning into their future teaching.

3) 90% of teachers will report increased knowledge and skills in such learning approaches.

METHODS

Program methodology and evaluation followed a concurrent, mixed-method approach to determine whether the Finding Home program met the objectives and outcomes as was successfully accomplished in previous Finding Home programs. Both quantitative and qualitative techniques were used for collecting data to increase confidence in the validity of the results. By using multiple types of data sources to increase confidence in results, the data combine the strengths and correct for deficiencies from any individual data source just as with previous Finding Home Programs. This is also known as triangulating the data. The evaluation included multiple data collection tools and both students and teachers participated. Evaluation tools included:

1) Student pre‐test and post‐test surveys with close‐ended, open‐ended and drawing item responses;
2) Student journal responses to prompts during instruction;
3) Final art student projects and the information students used to create their pieces;
4) Appreciation and attitude scales;
5) Student interviews, and;
6) Teacher interviews.

Student Pre‐Test and Post‐Test Surveys

The methods used for the 2013 program evaluation were similar to those used in previous years: 2009 (Dayer, Kemple, & Kilby, 2009), 2010 (Dayer, Kemple, & Meyers, 2010), and 2011 (Dayer, Kemple, & Meyers, 2011).

Student surveys followed a pre‐test/post‐test design. The single‐group pre‐test/post‐test evaluation design has been used extensively and has shown success in a variety of settings (Busch & Dayer, 2009; Weiss 1998).

Pre-surveys (Appendix A) were administered in January prior to all programming, and at the end of May (following all programming) to all classes and, thereby, to all students participating in Finding Home. It should be noted that some students were unable to take the surveys on the same day as the rest of the classes due to absences, and so they took the surveys after participating in the first program. Student pre-surveys were administered by the classroom teacher in the classroom and sent to the teachers by Coyote Trails staff following guidelines for administration (Appendix B).

Post‐surveys (Appendix C) were also administered by teachers, following guidelines for administration (Appendix D). The survey given to students after the program had the same types of questions as the pre-survey, but it should be noted that the wording of the questions changed to be more detailed and use program-specific academic vocabulary. These changes were made specifically to give students more context for their answers and encourage use of learned vocabulary and detail. The questions matched a theme of asking about specific knowledge we were trying to measure such as animal interactions, plant ecology, or human effect on natural areas.

Appreciation and Attitude Survey

The attitudinal survey items included two scales. The first scale, with three items, explored students’ nature‐related attitudes toward learning about nature. The second scale included three items that explored student behavioral interest in interacting with nature. The items in both scales were adapted from previous Finding Home programs.

Students were asked to what degree they agreed or disagreed with each attitude statement from 0 (strongly disagree) to 10 (strongly agree).

Attitude/Appreciation Scale

1. Nature is important to me.
2. Learning about nature is important.
3. I want to learn more about nature.

Attitude/Active scale

4. I don’t pay attention to things outside.
5. Doing things outdoors is boring.
6. I like to spend time outside.

There were three types of questions asked to students on the written pre- and post-assessments. These questions assessed program goals of increasing knowledge of nature and attitudes about the role of humans in the environment. A scoring guide was created for the drawings and open-ended responses based on rubrics previously used on Finding Home programs (Dayer et al., 2010).

Our assessment asked questions that prompted students to explain natural systems, interdependence, and interaction among humans and other natural elements living and non-living. Other questions assessed students’ knowledge of outdoor skills and early human history emphasizing the program goal of a human’s role in the environment. The degree to which the students could explain interactions and describe how to meet human needs in a natural environment lead to higher scores on a scale of 0 to 4:

No answer or cannot be determined – 0

One aspect or piece – 1

Two aspects or pieces interacting, connected or explained – 2

Three aspects or pieces interacting, connected or explained – 3

Two or three aspects or pieces with additional explanation on the interaction and its use of natural materials – 4

We used a basic rubric to assess students’ knowledge of basic animal tracking principles including foot morphology and gait patterns. This topic allowed students to gain detailed understanding and knowledge of animals and their interaction in wild, natural areas. Students were asked to draw their favorite animal’s track and gait pattern. Responses were then assessed using the following criteria adapted from rubrics used in previous Finding Home programs (Dayer et al., 2011). For each element, student responses received one point. The criteria assessed were: details of heal pad morphology, number of toe pads, claw register, and overall shape. Four points was the highest possible score.

Journaling Prompts

To assess their knowledge of the ecological role of humans in a wild habitat and interactions in the system, the students were asked to answer prompts in their personal journals after each unit of instruction. This helped instructors assess student understanding of material and gain a better understanding of each individual student. This information helped tailor future instruction to meet multiple learning styles as well as assist students in need of extra instruction.

Interviews

Student interviews were conducted by an instructor during the final two class visits and the field trip to the Coyote Trails Nature Center. A total of sixteen students, two from each class, participated in the interview. They were given the option to not participate, their names remained anonymous, and permission and release forms were sent home to parents for all students prior to their participation (Appendix E). Student interviews were conducted outdoors while other students were practicing or observing skills practice and while other adults and students were around but not listening, in order to ensure comfort by keeping an informal atmosphere.

Student interviews were semi-structured, using a script and allowing the interviewer to further encourage students to respond if necessary without giving them answers. Questions are shown below.

Interview questions

Question 1: What are the most interesting things you learned in the program you’ve been doing with us?
Wait for initial response, and then ask as a follow up: How does _______ tie in to natural cycles? How are the animals, plants, humans, and non-living things affected by this activity?

Question 2: What’s going on in this landscape? (Gesture to the natural area around)
Probes:
What kind of animals would live here?
What plants are helpful here?
What needs could you provide for here and how?
How are the living things around here connected?
What non-living things here are useful to humans or other animals?

Question 3: Would you be able to survive here? How would you provide for your four basic needs?

Question 4: What was your absolute favorite thing we did during the program?

Each student was asked individually, and it should be noted that student answers were not sound recorded; the interviewer only took notes on student answers. This may have caused students to be slightly uncomfortable in some cases. Before answering any questions, all students were asked if they were comfortable with the interviewer taking notes and were reminded their responses would only be used for a program report without the use of their name. Student responses were both reviewed and assessed by the interviewer.

Teacher Feedback

Teacher feedback questions were emailed to teachers directly. Teachers were instructed to respond via email or write responses on paper that we would pick up with the student post-assessments. The teachers were also reminded that their responses could be helpful for us to use as testimonial quotes when applying for other education-focused grants. They were reminded their names would not be linked to their responses.

Attendance Tracking

An attendance tracking sheet was kept by instructors at every field experience and art experience as well as the field trip to the Coyote Trails Nature Center. The sheet tracked individual students’ ability to participate in each visit.

RESULTS

Two hundred and seventy students participated in the program. Of these, all 270 completed both the pre- and post-test for both appreciation and knowledge surveys. From the results of the two questionnaires, there was an obvious and statistically significant increase in both attitude/appreciation and knowledge.

Attitude/Appreciation Survey

Specifically, with respect to the attitude and appreciation survey, the calculation of a t-value allows for the determination of certainty of the statistical significance. For differences between pre- and post-survey answers, given a sample size of 270, any t-value greater than 2.34 allows one to be 99% certain of statistical significance not attributed to error. Table 1 shows the calculated t-values. All t-values were found to be greater than 17.89. All of these t-values exceed the 2.34 minimum and prove that the differences between pre- and post-survey answers are meaningful. For example, question 1 of 10 had a pre-test average of 3.01 and a post-test average of 8.87. This shows an average increase of 5.86 points in student answers. This result, while also appearing to be a large increase, was proven to be significant by a t-value of 28.03. Figure 1 provides a graphical representation of the results shown in Table 1.

Table 1. Attitude/Appreciation Survey results with a student sample size of 270. The questions were scored on an 11-point Likert-scale from (0) being “strongly disagree” to (10) being “strongly agree”.

Students, then, showed an overall increase in appreciation for the natural world and a stronger desire to learn about and be in nature.

Table 2. Number of each answer given by a student for each question. For example, we can see that for question 1, a majority of students answered between 0 and 5 on the pre-test, however, a majority of students answered between 5 and 10 on the post-test.

Knowledge Survey

With respect to the (knowledge) survey, for differences between pre- and post-survey answers, given the same sample size as before of 270, any t-value greater than 2.34 allows the researcher to be 99% certain of statistical significance not attributed to error. In Table 3 below, t-values were calculated and all found to be greater than 15.93. All of these t-values exceed the 2.34 minimum and prove the differences between pre- and post-survey answers to be meaningful. For example, question 1 of 4 had a pre-test average of 0.62 and a post-test average of 2.60. This shows an average 1.97 point increase in student knowledge. This result while appearing to be a large increase was proven to be significant by a t-value of 25.69. Students, then, increased their knowledge of natural landscapes, a human’s use of materials, how that affects the area, and how the living and non-living elements interact. Figure 4 provides a graphical representation of the results shown in Table 3. Please refer to Appendices A and C for survey questions.

Table 3. Knowledge Survey results with a student sample size of 270. The questions were scored on a 5-point scale from (0) being “strongly disagree” to (4) being “strongly agree”. Questions 1, 6, 8, 11, and 12 from the survey were excluded from this statistical analysis as the answers to these questions were mostly opinions and difficult to translate onto a 0 to 4 scale.

Table 4. Number of each answer given by a student for each question. For example, for question 2, we can see that a majority of students answered either a 0 or 1 on the pre-survey, while a majority of students answered either a 3 or 4 on the post-survey.

Attendance was tracked during all instructional activities. Each heading relates to one of the nine visits: “skills” corresponding to days that students spent in the field learning hands-on skills and “art” corresponding to days that students received art-based instruction. The field trip is labeled as well. An attendance rate of 90% was the goal for all activities. This 90% goal was set to allow for weather cancellations, absences due to sickness, and other unforeseen circumstances. The lowest attendance rate seen was during Art3 during which 5 of the 270 students didn’t attend. All other activities had a higher rate of attendance with an average attendance rate for all activities being 98.97%.

Table 5. Number of absences and attendance rate per activity.

Selected Responses to Journal Prompts

In their journals, students were asked to write down things they observed using their “animal powers”. Animal powers are what instructors called the “heightened senses” activities taught early in the program. By bringing active attention to an individual sense, students would become more practiced in awareness and learn more about their landscape with their eyes, nose, ears, and touch. Figure 5 shows that the student isolated three different senses and documented heightened awareness.

Animal powers

Figure 5. Animal Powers A

Figure 6. Animal Powers B

Gait patterns

Students were taught three different gait patterns, and asked to respond to questions. Figures 7 and 8 show that these students could draw all three gait patterns, how many tracks are in a gait pattern, and shows that they knew that you can find out how fast an animal is moving from observing its tracks and gait pattern.

Figure 7. Animal Movement: Gait Patterns A

Figure 8. Animal Movement: Gait Patterns B

Foot morphology

In their journals, students were asked to study the structure of certain tracks and indicate unique parts of the animal’s foot morphology, as shown in Figure 9. This entry shows that the student knew the main differences between a feline and canine track, and more specifically why they are different.

Figure 9. Foot Morphology A

Figure 10 shows that this student knows the definition of foot morphology, and can differentiate between a canine and feline track both visually and with descriptive text.

Figure 10. Foot Morphology B

Mapping

In their journals students were asked to draw a map of the blindfolded string walk. This is an activity that challenges students to push their mapping skills an
d awareness skills using senses besides their eyes to navigate a course (by following a string with their hand and walking over and around obstacles in a natural area). The students were told ahead of time to pay close attention to sounds, experiences, emotions, smells, how the air felt, etc. in order to make a map afterward. Figure 11 shows that this student can draw a map make observations and locate landmarks that incorporate all their senses except sight in a natural landscape.

Figure 11. Mapping: Path

Wild edible plants

In their journals students were asked to take notes on the edible parts and uses of the plants known as the Big Four. These are four well-known plants in the survival and traditional skills community. They are easily found, useful in multiple ways, and edible. This was an introduction to plant ecology and human use/interaction with the plant world. Figure 12 shows that this student knew the uses and edible parts of pines, cattail, oaks, and grasses, and had an idea of how they could help in a survival situation.

Figure 12. The Big Four

Figure 13 shows that this student could list edible parts and other uses (e.g., tools, building material, fuel) for pine, oak, cattail, and grasses.

Figure 13. Observational Skills: The Big Four

Debris hut

Students were given a lesson on building debris huts. They were asked to write down all the things they could remember from our class. A debris hut is a shelter used by one person to keep warm, dry, and safe. It is made of sticks and dead forest debris with no need for any man-made materials. Figure 14 shows that this student could remember the names of all the parts of the debris hut, as well as how it would help with survival, plus the good and bad places to build it.

</s pan>

Figure 14. Debris Hut A

Figure 15 shows that this student knows where to build their debris hut, where not to build it, and all the parts that are needed to build it.

Figure 15. Debris Hut B

Bow drill fire

In their journals, students were asked to list as many things as they could remember from our class on making fire with a bow drill kit. This is a fire-making technique that uses a crafted wooden kit to make intense friction, and with practice and attention to the interaction of the body and the plant materials, makes fire. Figure 16 shows that this student could name and list all the parts needed to make a coal with a bow drill kit and displayed knowledge of what fire needs as well as the importance of friction.

Figure 16. Bow Drill Fire A

Students were asked to draw and label the different parts of a bow drill kit. Figure 17 shows that the student knows all the parts of a bow drill kit and what they look like.

Figure 17. Bow Drill Fire B

Contour line drawing

Students were asked to participate in a blind, contour-line drawing exercise. This is an introductory activity used to teach students to draw from observation rather than just “tracing.” It is a practice used to look for shapes and curves rather than be overwhelmed by an entire subject that you want to draw. Figure 18 shows that this student was successful at seeing minute details and communicating them in drawing.

Students were also asked to participate in a blind, contour-line drawing exercise designed to help them draw a complicated image from a photo and help them see details and br
eak down shapes. Figure 19 shows that this student was successful at breaking down a complicated image into line and shape.

Figure 18. Contour Line Drawing A

Figure 19. Contour Line Drawing B

Art Project Assessments

Plant art day

Students were asked to complete an art project reflecting on our lessons on the Big Four, animal powers, observations skills, and drawing. The requirements for the art project were to have a large detailed drawing of one of the plants from the Big Four and to have at least five observations or uses of the plant they chose. Figures 20, 21, and 22 are examples of the students following directions and clearly communicating their knowledge and observation of the plants they chose. The drawings show the students were successful at noticing and communicating details they observed visually.

Figure 20. Pine Needles

Figure 21. Cattails

Figure 22. Oaks

Animal art day

Students were asked to complete an art project reflecting on the lessons in foot morphology and gait patterns. The requirements for this project were to pick an animal that lives in Oregon and to create three drawings, one of the animal they chose, one of its tracks, and one of its gait pattern, then to l
ist at least five observations they made about the animal. Figures 23, 24, and 25 are good examples of students following directions and showing knowledge of the difference between a track and a gait pattern, displaying their understanding of observations and making informational detailed drawings that supported their knowledge.

Figure 23. Coyote

Figure 24. Cougar

Figure 25. Rabbit

Mapping art day

Students were asked to complete an art project that reflected on the blindfolded string walk and mapping lessons. The requirements of the project were to draw a detailed map of their favorite place, to include why it was their favorite place, and to include a key. Figures 26 and 27 are good examples of students following directions, employing their knowledge of a bird’s eye view, and using symbols in a key and scale. It is also a good example of using memory to recall details and using drawing skills to help communicate them.

Figure 26. Map of Home A

Figure 27. Map of Home B

Skills art day

Students were asked to complete an art project reflecting on the lessons in traditional skills throughout the progra
m including debris hut, bow drill fire, wild edibles, and throwing stick (a traditional hunting technique used to teach heightened awareness and animal movement). The requirements of the project were to pick a skill and to use text and drawing to create an informational poster teaching others how to employ the skill and why they would need it to survive. Figures 28, 29, and 30 are good examples of students following directions, knowing the parts and drawing them, showing step-by-step directions of the process, and using their knowledge to help others learn.

Figure 28. How to Make a Debris Hut

Figure 29. How to Use Pine Needles

Figure 30. How to Use a Bow Drill Kit

Student Interviews

Student interviews were conducted to obtain a sampled selection of individual experience within the program. Sixteen students were randomly selected for the interviews. Then an evaluation was conducted to compare different individual answers and assess results in conjunction with pre- and post-survey information. This triangulation of data helped determine validity in the measured outcomes. The interviews included three open-ended questions to which students responded; they were also prompted to discuss their answers in more detail. Questions were chosen in order to allow the student to talk about what he/she was interested in relating to the natural world and this program. The students were interviewed one at a time while still around other students and teachers. Those near the interview area were not listening, but engaged in an activity. Students were given the option to decline being interviewed; none declined.

Question 1: What are the most interesting things you learned in the program with us?

The first question was intentionally broad and was intended to transition students into remembering the past few months of instruction and the different lessons involved. The students were asked to tell the interviewer one or two things that they found the most interesting from all the activities they had experienced in the program. All students gave specific activities, except two who said that all of them were equally interesting. Some students gave two answers, and some gave only one answer, so there are more than sixteen responses accounted for in the data (see Figure 31).

Movement-related responses refer to stealth techniques that were intended to help students move slowly outdoors and allow for more detailed observations of their surroundings. Shelter responses refer to short-term and long-term shelters made from natural materials emphasizing how and why the shelter works to keep a person or animal warm and dry. Fire responses refer to primitive techniques to create a fire that has multiple uses using only natural materials found on the landscape. Plant responses refer to wild edibles, invasive species, conservation of riparian zone plants, and plants harvested to make tools or containers in a traditional manner.

Once the initial response was recorded, students were asked to go more in depth by tying their response to the idea of natural cycles and human interaction with the natural world. Answers that were described an interaction with plants, animals, non-living things, and the effects of the practice over an extended time were scored as a 4; three of those elements scored 3; two elements, a 2; one element, a one; and no answer or no elements, a 0.

Question 2: “What’s going on in this landscape?”

This question was asked at the Coyote Trails Nature Center in Medford, Oregon. It was made clear to each student that he or she was supposed to identify as many interactions and useful plants, animals, or other non-living things that could be used for human survival as they could observe. The interviewer was also permitted to ask follow-up questions and prompt students to investigate further. Among the detailed answers were shelter materials, fire-making materials, birds, small animal homes, riparian area, shelter locations, container materials, animal tracks, flint knapping tools, cordage materials, and human effects on the area such as tracks and plant use.

Question 3: “Could you survive here?”

Each student was also asked to explain his or her answer. This question was asked to help measure students’ comfort with the hands-on skills that they had been taught through their experiences in natural settings, providing for their own needs using only natural materials.

The students were prompted to explain a way to provide shelter, safe water, fire, and food in a wild setting. Any answers that considered the Nature Center or surrounding stores and houses were re-prompted to imagine they did not exist. The question was rephrased, “If you had no access to any of the stores, houses, or the Nature Center, how would you provide for your four basic needs?”

Responses that pointed out the materials the student would use and explained what he/she would do for each of the four needs received an 8. Scores with less detail, attention to materials around him/her, and lack of clear understanding declined from 8 all the way to zero. However, no student’s answer was scored below an 8. All 16 students were able to give a detailed explanation of the materials they would use, how it would affect the plants and animals in the area and the system as a whole, and extra details about how the skill is accomplished (e.g., shelter building, fire by friction, traditional hunting, and primitive water filters).

Teacher Interviews

Teacher interviews were conducted to enable evaluation of the program’s teacher objectives as well as to provide a formal outlet for teachers to give feedback . Teachers were asked six questions. Due to scheduling, teachers and interviewers were unable to meet in person, so the questions were given to teachers via an attached document in an email. Teachers were asked to fill out the interview questions and return to the interviewer by a deadline. Responses were not quantified, but answers have been paraphrased below.

Question one: Was there anything the instructors did during instruction that stood out to you (positive or negative)?

Teacher feedback for this question was all positive with some suggestions for improved instruction. Teachers reported that instructors were motivated, skilled, enthusiastic, effective, and engaging. The lessons were relevant, fit together, delivered well, and unique.

Suggestions included using more science-based vocabulary throughout the lessons.

Question two: Were you able to incorporate the Coyote Trails activities or topics into your classroom or instruction?

Seven out of eight teachers said they were able to incorporate Coyote Trails instruction into their science and art curriculum. “[It] tied in perfectly,” and “[It] replaced our science time on the days you were there.” One teacher wrote, “It would be ideal to have the plan ahead of time so that I could tie the topics in better to our science lessons ahead of time.” This teacher also suggested a wider range of art mediums be used in order to tie in her own instruction better.

Seven out of eight teachers reported becoming more comfortable with the methods behind incorporating nature/place-based education into regular classroom instruction. One teacher reported already being comfortable with incorporating outdoor learning experiences into her classes, but that she had gained more ideas and was “inspired to learn even more on her own.” This feedback was counted in favor of the objective to increase comfort with incorporating outdoor education into normal classroom instruction, making the result eight out of eight teachers.

Four teachers of eight reported wanting to try to teach something similar themselves. Three teachers actually did plan outdoor experiences after participating in the Finding Home program and credited their knowledge and ability to do so to their observation of the program itself. This was interpreted as seven of eight teachers with increased skill in incorporating outdoor instruction into their teaching.

Question three: Is there anything we could do, in your opinion, to improve the program (scheduling, number of instructors, content, etc.)?

Two of eight teachers expressed a desire to extend the time of each classroom visit, “Some kids didn’t finish projects.” Other feedback included the suggestion of providing vocabulary sheets. Two of eight teachers wrote that they would make no changes. One teacher suggested more use of other art mediums “such as clay, natural dyes, or 3D art.”

Question four: Are there any positive stories you would like to share about particular students or your class overall?

All eight teachers reported seeing improvement in certain or all students’ engagement with the material. “There were a few particular students I saw who would never have engaged in science material like they did working with this program.” One teacher shared this: “I would bring Coyote Trails to my classroom every day if I could. The kids have plenty of stories!” Five out of eight teachers mentioned students having fun while doing science and art material.

A favorite story:

“My student [name removed], told me that she would never play outside. It was dirty and she was absolutely scared of getting hurt or bitten or attacked. The Coyote Trails teachers only got to see her tantrum on the first day, but for the week leading up to their visits, [the student] would complain of injuries and headaches. She was fine, and on the days they would visit, she would wear a dress or high heels hoping to get out of the activities. The teachers didn’t even address it. It was a totally normal thing, and it was clear they were prepared to help students feel more comfortable outside. Their experience was obvious, and [the girl] was on her hands and knees, in a dress, with no shoes on, sneaking through the grass with the rest of her classmates. She was the first one to volunteer answers during review, and now even pays more attention in class overall. I am very happy with the outcome of this program.”

Question five: Do you think outdoor education is important? Why/why not?

Eight out of eight teachers said yes. Some answers included reasons such as kids having limited access to the outdoors, school being a place for exploration which is the key to education, hands-on experiences leaving long-lasting impressions, and being a fun way to learn about things.

One teacher wrote that outdoor education should be a top priority in schools because it is so effective. All eight responses addressed gaining a deeper understanding of how to integrate outdoor learning into science class specifically, but two of eight teachers mentioned art as well. “I understand what other teachers who have done this program are talking about now. Outdoor learning is a tool and sometimes essential to get students to understand that what you’re talking about lives right next to them.” One teacher suggested more workshop time for teachers specifically, noting that getting the instructional plans ahead of time would help teachers assess the process of teaching outdoor-specific lessons.

Question six: Do you think students in your class appreciate nature/outdoors more since being in the Finding Home program? If yes, how can you tell?

Eight out of eight teachers answered yes. The use improved energy, engagement, and intrigue when outside or learning about science as evidence.

One teacher wrote, “I’m not sure they ever thought about [nature] much before, but now I always see them looking around or saving a bug or something. My kids had a life-changing experience I think.”

DISCUSSION

By assessing results from the multiple evaluation tools employed, it was possible to determine the overall effectiveness of the Finding Home program and compare results to our original objectives.

Objective 1
Ninety percent of the eight classes of 4th/5th grade students will take part in all five field experience, four classroom visits, and contribute one of each student’s art products to the community event.

Result 1
In Table 5 above, it can be seen that student attendance never reached below 98.15% for an individual activity; therefore, at least 90% of the eight classes of students took part in all five field experiments, four classroom visits, and contributed to the community event.

Objective 2
Student knowledge about the natural system will increase significantly (at the .05 level) from pre- to post-surveys.

Result 2
As depicted in Table 3, the t-values calculated (ranging from 15.93 to 71.72) all exceeded the 1.65 minimum for 95% (at the .05 level) certainty; proof that student knowledge about the natural system increased significantly from pre- to post-surveys.

Objective 3
80% of interviewed students will demonstrate increased knowledge of the natural systems.

Result 3
Table 6 shows the results by question of the knowledge pre- and post-surveys given to a small sample of the students. In this sample population, it can be seen that, for example, student 1 had a pre-survey total (summation of answers given) totaling 4 points. On the post-test this student totaled 20 points, and because a larger value given in the answers demonstrates an increased knowledge of the natural systems, this table shows that for students 1 through 10, all demonstrated increased knowledge of the natural systems. This same trend was seen throughout, and all 270 students (100%) demonstrated increased knowledge, surpassing the minimum 80%.

Table 6. Sample of 10 students depicting 100% improvement by students, surpassing the minimum of 80% needed.

Objective 4
Student appreciation of the natural system will increase significantly (at the .05 level) from pre- to post-surveys.

Result 4
As depicted in Table 1, the t-values calculated (ranging from 17.89 to 35.30) all exceeded the 1.65 minimum for 95% (at the .05 level) certainty; proof that student appreciation about the natural system increased significantly from pre- to post-surveys.

Objective 5
85% of interviewed students will demonstrate increased appreciation of the natural world.

Result 5
Table 7, below, shows the results for a sample of 10 selected students. A higher score on the post-test for questions 1, 2, and 3 showed an increase in appreciation, while a higher score on the pre-test for questions 4, 5, and 6 showed an increase in appreciation of the natural world. The improve column holds a “1” if for that student, the overall score for increased appreciation surpasses the overall score for decrease in appreciation. For these 10 students, all showed an increase in appreciation of the natural world, and this trend follows for all 270 students (100%) in the survey, surpassing the minimum of 85% needed.

Table 7. Sample of 10 students depicting 100% improvement by students, surpassing the minimum of 85% needed.

Teacher Outcomes

Objective 1
90% of teachers will report increased comfort with including experiential, outdoor focused, place‐based, and inquiry‐based learning in their curriculum.

Result 1
100% of participating teachers repo
rted increased comfort with including outdoor education as a part of their regular instruction plans throughout a unit or academic year.

Objective 2
90% of teachers will report increased likelihood to integrate such learning into their future teaching.

Result 2

Seven out of eight teachers reported increased likelihood to integrate similar instruction into their regular academic year. While this is only 88%, the teacher who did not give this response also did not report the contrary.

Objective 3
90% of teachers will report increased knowledge and skills in such learning approaches.

Result 3
One hundred percent of participating teachers reported increased knowledge of outdoor instruction, but it was suggested that teachers could benefit from additional resources to increase their knowledge.

CONCLUSION

The Finding Home program exceeded expected outcomes for the participating 4th and 5th grade students from four Title I schools in Southern Oregon.

This year’s program included fieldwork both at the local school and at the Coyote Trails Nature Center. Focused measures of the students’ appreciation and knowledge show significant increases in understanding of and connection with nature. It is suggested, however, that new survey measures be developed to provide more quantifiable results. Likewise, teachers commented that they felt students’ attitudes toward their local natural world had changed. The results indicate that student connection to natural landscapes increased. It is suggested that there be more pre- and post-assessment of teacher attitudes and comfort in order to make results more reliable.

Students’ knowledge of nature topics also increased greatly according to multiple measures. The use of interviews, analysis of drawings, and pre- and post-program surveys provided triangulated data that strongly shows program impacts. Further, there may still be the opportunity to further enhance students’ understanding of how elements of the environment influence each other. All students demonstrated that they learned how to produce products that linked nature learning with creative art, but in the future we suggest a stronger link between the art projects and field experiences. One possibility would be to use natural craft, traditional skills, or natural materials to actually make a project (baskets, flint knapping, natural dyes).

The expected outcomes for teacher participants also showed success and additionally included suggestions for future improvements. The teachers enhanced their knowledge, skills, and comfort with a place-based education approaches to environmental education. However, due to a lack of pre-survey data, it is hard to quantitatively or accurately assess the effect of the program on teacher’s skill, knowledge, or comfort. In the future, we suggest using the same pre- and post-surveys for teachers in order to have comparable data for those specific objectives. We suggest creating a teacher-focused day that includes a discussion about outdoor education in general. This would facilitate greater interchange between public school teachers’ and outdoor educators’ perspectives.

Outdoor education programs typically require significant teacher training in order to introduce new and challenging pedagogies such as place‐based learning. Regardless of the definitional challenges, all teachers expressed a new comfort with using local natural places on their own for teaching.

In addition to demonstrating the beneficial outcomes of more intensive nature-contact programming, this evaluation highlights how a multiple methods approach to evaluation can offer multi-faceted insight into a program. Triangulation with these methods again provided a rich, although sometimes complicated, picture of the outcomes of the program. It is also beneficial to look at the qualitative and written responses of individuals from a sample of participants in order to truly connect with the living outcomes of the program. Without this qualitative data, we believe the evaluation cannot present an accurate picture of the results. The use of multiple methods also provides very helpful insights into what evaluation methods might be improved, such as the suggestion for an improved teacher and student survey.

This Finding Home program evaluation will be increasingly valuable to the advancement of knowledge if the evaluation tools and methods created for this program are used again in future Finding Home programming and other programs to provide a basis for comparison.

Bibliography

Dayer, A., Kemple, L. & Kilby, A. (2009). A comprehensive evaluation of the “Finding Home”

pilot program. Report prepared for the Oregon Community Foundation. Medford, OR:

Jefferson Nature Center.

Dayer, A., Kemple, L. & Meyers, R. (2011). A Comprehensive Evaluation of the “Finding Home” Program: Year Three. Report prepared for the Oregon Community Foundation. Medford, OR: Jefferson Nature Center.

Dayer, A., Kemple, L. & Meyers, R. (2010). A comprehensive evaluation of the “Finding Home” program: Year Two. Report prepared for the Oregon Community Foundation. Medford, OR: Jefferson Nature Center.

APPENDIX A

Pre-Survey

What is your first name and last initial?

What is your teacher’s last name?

Answer the questions below. If there is something you don’t understand or a word you don’t know, circle the question or word. If you need, you can answer on the back too.

1. Draw your favorite place to be when you’re not inside. Why is it your favorite place?
2. What does it mean to live primitively or traditionally?
3. What is your favorite animal? Draw the track of your favorite animal. (Bonus: Do you know its gait pattern?)
4. Can you list the any wild, edible plants? What are the uses for one of them (pick one to tell us about)?
5. Your basic needs are shelter, water, fire, and food. Name one way to provide for each of them using natural materials.
6. How many times do you play outside during the week? What are you usually doing?
7. What kind of tools did Early Native Americans use? List three
8. Is learning about nature important? Why or why not?
9. How would you keep warm in the winter without a house? If you know a certain technique, explain it please.
10. How would you start a fire without matches or a lighter in order to cook your food? If you know a certain technique, explain it please.
11. Are you excited about working with Coyote Trails? Why or why not?

APPENDIX B

Pre-Survey Directions

Please…

-give students plenty of time to respond completely to each question.
-feel free to help students who are struggling remember to just do their best, but please do not help them by prompting answers.
-Have students remain silent until everyone finishes the evaluation.

APPENDIX C

Post-Survey

Answer the questions below. If there is something you don’t understand or a word you don’t know, circle the question or word. If you need, you can answer on the back too.

• Draw your favorite place to be when you’re not inside. Why is it your favorite place?
• What does it mean to live primitively or traditionally?
• What is your favorite animal? Draw the track of your favorite animal. (Bonus: Do you know its gait pattern?)
• Can you list the Big Four? What are the uses for one of them (pick one to tell us about)?
• Your basic needs are shelter, water, fire, and food. Name one way to provide for each of them that you learned from Coyote Trails.
• How many times do you play outside during the week? What are you usually doing?
• What kind of tools did Early Native Americans use? List three.
• Your art projects were fantastic and they help people learn about nature! Is learning about nature important? Why or why not?
• How would you keep warm in the winter without a house? If you know a certain technique, explain it please.
• How would you start a fire without matches or a lighter in order to cook your food? If you know a certain technique, explain it please.
• Did you enjoy working with Coyote Trails? Why or why not?
• Have you taught anyone else the skills you learned with Coyote Trails? Who?

APPENDIX D

Post-Survey Directions

As we begin to wrap up writing our evaluation of the Finding Home program, our data collection will include the pre-assessments the students first took, journal prompts, personal casual interviews, art projects, and post-assessment material.

The post-assessment is attached and is not exactly the same as the pre-assessment. We hope you can find time during your busy end-of-the-year to have students fill these out; it is pertinent to our follow-up and grant evaluation. Please feel free to fill one out yourself too!

Please…
-give students plenty of time to respond completely to each question.
-feel free to help students who are struggling remember to just do their best, but please do not help them by prompting answers.
-have students remain silent until everyone finishes the evaluation.

There is another document attached which is specifically for teachers participating in an extended CT program. We really appreciate the insightful feedback from teachers to help improve our programs. This kind of feedback from educators and administration will help us gain traction in the valley with other programs for schools. We can use your feedback on future grants for similar programs and continue developing a better, more in-depth program for classrooms.

Please…
-answer all questions completely.
-don’t worry about hurting anyone’s feelings; this feedback is essential.
-feel free to add any notes that you may have that do not fit an included question.

APPENDIX E

2013 Permission and Release

I hereby grant permission for the below named Participant to participate in a field trip with Nature Awareness and Wilderness Sports Programs dba Coyote Trails School of Nature (“NAWS”), an Ohio non-profit corporation that is qualified as a foreign corporation in the State of Oregon. I understand that NAWS’ activities take place on property that may include certain risks and that the NAWS’ activities themselves may involve inherent risks. The Participant releases to NAWS the rights to use any photograph or video taken while participating in said activity, to be used as deemed by CTSN, including advertising.

The terms of this Permission and Release shall be governed by the internal substantive laws of the State of Oregon without giving effect to any choice or conflict of law provision that would cause the application of the laws of any other jurisdiction.

Participant’s Printed Name __________________________________________________ Date: __________________________

Signature of Participant (if over 18 years of age)

Or Parent or Legal Guardian of Participant: __________________________________________________________________

Parent and/or Legal Guardian Printed Name _____________________________________________________________________

2013 Permiso y Liberación

Por la presente doy permiso al participante, nombrado abajo, para participaren una excursión con Nature Awareness and Wilderness Sports Programs dba Coyote Trails School of Nature (“NAWS”), una organización sin ánimo de lucro de Ohio, cualificada como corporación extranjera en el Estado de Oregon. Comprendo que las actividades realizadas por NAWS tienen ciertos riesgos y en lugares que pueden incluir ciertos riesgos. El participante concede el derecho a NAWS de usar cualquier fotografía o video tomado durante su participación en las actividades, según es considerado por la CTSN, incluido el uso publicitario.

Los términos de este Permiso y Liberación se regirán por las leyes internas del Estado de Oregón, sin dar lugar a elección o conflicto legal que se pudiese derivar de la aplicación de las leyes de cualquier otra jurisdicción.

Nombre del participante: __________________________________________________ Fecha: __________________________

Firma del participante (si es mayor de 18 años)

O padre/ tutor le
gal del participante: __________________________________________________________________

Nombre del padre/tutor legal del participante: _____________________________________________________________________