Tests-Inauthentic Methods for Measuring Learning.

July, 2017

Elliot Eisner said,  “Not everything important is measurable and not everything measurable is important.”

Middle School: The Flat Years?

When my children were in middle school.  I read a piece that stated standardized test scores are flat or depressed for many middle school students.  What strikes me most about this is that I have witnessed perhaps the greatest level of intellectual growth during these “flat” middle school years.  I’ve noticed middle-schoolers have strong feelings about more abstract concepts such as social justice; they recognize the nuances of relationships, they can verbally express desires and fears, and they argue their position with reasoning although sometimes faulty.  I see metaphoric thinking and hear more questions about how the world works.  If anything, learning has been exploding during these years; learning is hardly dormant as scores might suggest.  My sense is that standardized tests cannot tap into that kind of learning.  It certainly cannot measure it.

 Watching Learning Happen While It's Happening

    It’s hard not to think of my own learning as an adult; the kind of learning that takes place everyday as part of being alive and a member of the human race.  We are all constantly learning and we can choose to do this formally and intentionally.  Although more often, learning  happens accidentally. (Examples: navigating the subway system in New York City, learning to body surf, making a new recipe, etc) In Creative Schools, author Ken Robinson enlightens, “The lives of most people have not followed a standard course. People commonly move in unexpected directions, discover new interests, or take unplanned opportunities.” And learning occurs because of it.

    Just a few days ago, my daughter learned to play the Beatles’ song  “Here Comes The Sun” on her guitar.  She taught herself. As she was working through it, I suggested she might want to try using a You-tube video.  She assured me that it was just easier to experiment on the guitar and try different notes until she got it right.  Either way, she didn’t need school or a teacher to help her learn (although perhaps she would have gotten it sooner but I believe the fun for her was figuring it out herself) and she certainly didn’t need an assessment to prove she had learned it.

 

“If true learning is enduring and meaningful, I can say this about tests- I have learned many things in my life and never needed a test score to prove it.  Conversely, I have passed many tests and never learned a damn thing.” (Roseann Cetta, 2017)

Book Review: Ken Robinson's Creative Schools

 Education is not the filling of a pail, but the lighting of a fire.

(William Butler Yeats)

Young children arrive to their first day of kindergarten with a healthy wick from which a fire can be lit.  Before they leave elementary school however, many children are bored, unmotivated, dislike school, feel inadequate in at least one subject area and, most disappointing, have lost a love of learning.  School has become a thing to get through, a place to see friends, and on good days, fill their pails with a thing or two.  Children are not going to school to develop their minds.  But, that’s exactly what a school should be doing.  Educating students means helping them to develop their minds, inspire the love and learning, and teaching them how to learn.

 

 Schools: The First Spirit Killer

Reading Creative Schools has been a game changer for me.  The intentions of the Tough Times report as well as the Common Core Content Standards born of that report are both valid and compelling. But the line continued past theories of learning, pedagogy, child development, and went straight into some common sense business, industry, factory model leaving out the human variable component of schools: our children. The CCCS moved in the completely wrong direction and away from the light, surely away from any fire lighting.  Robinson writes, “Although all the rhetoric of the standards movement is about employability, the emphasis has not been on courses that prepare people directly for the work but on raising standards in academic programs.”  He also notes, “Few if any of the abilities that entrepreneurs need are facilitated by the strategies that reformers value so much. On the contrary, standardized education can crush creativity and innovation, the very qualities on which today’s economies depend.”  In addition and most importantly, the standards movement ironically is making worse the exact thing it is trying to fix.  “The standards movement is not achieving the objectives it has set for itself.  Meanwhile, it is having catastrophic consequences on student engagement and teacher morale.”

What school must feel like for so many of our children?

Imagine administration sharing this with staff.

         “Ok teachers- new goal.  We have decided that it is very important that every one of you will run a marathon.  You have a year to train.  The school will provide you with a coach and running shoes!”  Now, Ms. Jones over there is pretty excited, and so is Ms. Smith.  They are runners.  They will excel.  They think this is a great idea.  Mrs. Connor. is not at all enthused.  She’s 65 and her knees are not in good shape.  She thinks perhaps this is the time to retire. Several other teachers start a different kind of panic.  They cannot retire yet as they need their jobs. 

A few rebels in the back raise their hands.  “Ummmm I don’t really want to run a marathon.  What is the point?” they ask. 

“Oh the point is to help your health.”   “Running is good for you. The world is a very unhealthy place and running this marathon will get your heart in shape, improve your joints, the high impact will help with osteoporosis and you’ll even walk away with a lifelong hobby that you can take with you into your personal life.” 

Another teacher asks, “Why 26.2 miles?  Can’t we just do 3 miles?” 

“That’s called having low standards.  We need to have high standards. The highest.”

After the training starts, it becomes immediately clear who is going to need extra special help.  Several basic skills groups are formed.  There’s even a very special group for marathon-disabled runners. Motivation is low and morale is even worse. 

One of the so called “runners’ in the low group asks a question.  “Can we walk?  I enjoy walking and I think I can walk a far distance,” she says, “I know it’s not the same as running a marathon, but it is still good exercise.”

Another teacher asks about yoga.  She said she’s actually good at that and believes there are benefits to that activity as well.  There are a few other folks who have decided to drop out of the whole process but will make music and signs for those who are still excited about running…but they are viewed as failures.

According to Robinson, “By conformity, I mean the institutional tendency in education to judge students by a single standard of ability and to treat those who don’t meet is as “less abled” or “disabled” –as deviations from the norm.

Finally, someone from the lower groups asks a very good question,  “Is running a marathon the only way for us the achieve our goal?”

 

Standards-

The standards movement is destroying the very thing it set out to improve

Teachers and administrators and education policy makers have bought into this notion that students who are not achieving the levels of our even now higher standards in the areas of language arts and/or math are failing students.  Because that is what the system is telling them.  Because Armageddon is coming if student’s strengths aren’t in the area that is being tested. Talents and skills come in many forms.  But not in schools.  Children in the elementary school do learn one thing quickly; who has what it takes to succeed in school and who doesn’t.  No wonder so many children lose their natural curiosity, their desire and openness to learn.  No wonder the flame is extinguished.

 

How Does Society Work?

Marathon running is another good metaphor as we consider how society works.  It’s an individual activity: one a person can choose to do and succeed at on her/his own terms.  It’s competitive too perhaps with others but always with oneself.  Society is much more similar to a softball team.  Everyone on the team has different talents.  It’s true that it will be good if everyone can hit the ball, catch, and throw: the fundamentals.  But our shortstop must have quick reflexes and a strong throwing arm. Our weaker hitters can always bunt.  Our fast runners can play in the outfield and are great base runners.  Some teammates will act as a pinch hitter, and every team needs a relief pitcher. All skills and strengths have value.  And while, it would be wonderful if every player were greatly skilled at every aspect of the game, that’s just not how it works. We have lost that idea in schools as well.

Robinson notes, “First, all students have great natural abilities.  Second, the key to developing them is to move beyond the narrow confines of academicism and conformity to systems that are personalized to the real abilities of every student.” “…. our school systems are now a matrix of organizational rituals and intellectual habits that do not adequately reflect the great variety of talents of the students who attend them”

Robinson also points out,  “Our communities depend on the enormous diversity of talents, roles, and occupations.  The work of electricians, plumbers, chefs, paramedics, carpenters, mechanics, engineers, security staff and the rest (who may or may not have college degrees) is absolutely vital to the quality of our lives.  Very many people in these occupations enjoy them enormously and gain great fulfillment from them. One effect of the emphasis on academic work in schools is that the education system is not focused on these roles and typically considers them second-rate options for people who don’t make the academic cut.”

 

 

Places of Learning vs. Schools

What Dewey theorized and observed in his lab-schools have been proven with modern science. Learning is a natural process through experiences when the mind makes meaning of the event. Children, especially young ones, can easily be observed learning through play and other activities.  Curiosity leads to exploration, and exploration leads to new information and skills to pre-existing schema and so on.  Learning is most motivating when it is meaningful, relevant, and personal.  Most schools function on some outdated notion that information can be pushed into the mind of a child.  Robinson uses a garden metaphor to demonstrate better a teacher’s role in student learning.

“Gardeners know that they don’t make plants grow. They don’t attach the roots, glue the leaves, and paint the petals.  Plants grow themselves.  The job of the gardener is to create the best condition for that to happen.  Good gardeners create the conditions for learning, and poor ones don’t.”

Schools and teachers in particular can effectively use this model with the right supports from administrators, a willingness to create learning experiences, and permission to let go of old ways of doing things. Of course assessment is assessment and it’s not going away anytime soon. One conclusion we can draw is that despite enormous resources, both financial and human, most standardized scores are not rising.  Even though teachers have been told to raise them and their own evaluations are tied to their students’ performances, the needle hasn’t moved far.  So how to move away from the old and into a new place where students can explore and learn and contribute their talents and strengths and still be accountable for learning what will be “on the test”?

 

Assessments: Tests-Inauthentic Methods Measuring Learning.

Elliot Eisner said,  “Not everything important is measurable and not everything measurable is important.”

When my children were in middle school.  I read a piece that stated standardized test scores are flat or depressed for many middle school students.  What strikes me most about this is that I have witnessed perhaps the greatest level of intellectual growth during these “flat” middle school years.  I’ve noticed middle-schoolers have strong feelings about more abstract concepts such as social justice; they recognize the nuances of relationships, they can verbally express desires and fears, and they argue their position with reasoning although sometimes faulty.  I see metaphoric thinking and hear more questions about how the world works.  If anything, learning has been exploding during these years; learning is hardly dormant as scores might suggest.  My sense is that standardized tests cannot tap into that kind of learning.  It certainly cannot measure it.

 

It’s hard not to think of my own learning as an adult; the kind of learning that takes place everyday as part of being alive and a member of the human race.  We are all constantly learning and we can choose to do this formally and intentionally.  Sometimes learning  happens accidentally. (Examples: navigating the subway system in New York City, learning to body surf, making a new recipe, etc)  Robinson enlightens, “The lives of most people have not followed a standard course. People commonly move in unexpected directions, discover new interests, or take unplanned opportunities.”

Just a few days ago, my daughter learned the Beatles’ song  “Here Comes The Sun” on her guitar.  She taught herself. As she was working through it, I suggested she might want to try using a You-tube video.  She assured me that it was just easier to just play it on the guitar and try different notes until she got it right.  Either way, she didn’t need school or a teacher to help her learn (although perhaps she would have gotten it sooner but I believe the fun for her was figuring it out herself) and she certainly didn’t need an assessment to prove she learned it.

 

“If true learning is enduring and meaningful, I can say this about tests- I have learned many things in my life and never needed a test score to prove it.  Conversely, I have passed many tests and never learned a damn thing.” (Roseann Cetta, 2017)

 

Robinson discusses the need for more comprehensive, authentic ways to measure learning. He talks about students reflecting on their own learning and self-assessing.  If we value self-learning, we should value self-assessment and, even more so, self-reflection. 

As a teacher, I always included a blank lined page at the end of a test.  It was the place for students to write what they learned that wasn’t already on the test.  As you can imagine, reading what students wrote in this section was enlightening.  It also provided an opportunity to give some extra credit and boost the grade especially if it moved a student into a passing grade. 

“In national and global surveys, employers don’t complain about applicants lacking specific knowledge or technical skills, which are easy to test and express in a letter grade; they want employees who can analyze critically, collaboratively, communicate, solve problems, and think creatively.

 

Changing schools and lighting the fire for children’s learning is not impossible. It will require a massive paradigm shift but until the movers get moving to make that happen, teachers can make small changes within their classrooms to begin focusing on their students’ strengths and natural abilities even if they are not annually tested.

Robinson notes, “We could encourage the best qualities of youthfulness—curiosity, adventure, resilience, the capacity for surprising insight—simply by being more flexible about time, tests, and texts, by introducing kids to truly competent adults and by giving each student autonomy he or she needs in order to take a risk every now and then.”  “It’s about making sure that we don’t let kids go through school not knowing what their strengths are.”

No more filling buckets! Time to light up those wicks.

 

Places of Learning vs. Schools

 Places of Learning vs. Schools

What Dewey theorized and observed in his lab-schools, modern science has proven to be true. Learning is a natural process; through experiences, the mind makes meaning of the event. Children, especially young ones, can easily be observed learning through play and other activities.  Curiosity leads to exploration, and exploration leads to new information and skills, this new information and these new skills get added to pre-existing schema and so on.  Learning is most motivating when it is meaningful, relevant, and personal.  Most schools function on some outdated notion that information can be pushed into the mind of a child.  Robinson uses a garden metaphor to demonstrate better a teacher’s role in student learning.

“Gardeners know that they don’t make plants grow. They don’t attach the roots, glue the leaves, and paint the petals.  Plants grow themselves.  The job of the gardener is to create the best condition for that to happen.  Good gardeners create the conditions for learning, and poor ones don’t.”

Schools and teachers in particular can effectively use this model with the right supports from administrators, a willingness to create learning experiences, and permission to let go of old ways of doing things. Of course assessment is assessment and it’s not going away anytime soon. One conclusion we can draw is that despite enormous resources, both financial and human, most standardized scores are not rising.  Even though teachers have been told to raise them and their own evaluations are tied to their students’ performances, the needle hasn’t moved far.  So how to move away from the old and into a new place where students can explore and learn and contribute their talents and strengths and still be accountable for learning what will be “on the test”?

Dewey Knew: How Cognitive Science Corroborates John Dewey's Observations About Learning Especially Those Middle Years

 From Dewey at the turn of the century to modern neuroscience, understanding how the brain learns seems to me as the most fundamental information every teacher should know.  What Dewey hypothesized based on observations, modern neuroscience has confirmed. How interesting that the two most important theorists on learning are 100 years apart, and to me, equally relevant.   

Dewey

Based on observations in his lab school, Dewey explained that a learner must be an active participant in the learning process and people (not just children) need experiences to learn.  This contrasts ideas both before and after his work where it was believed that information could be more passively transmitted into the learner without the learner’s active involvement.

According to Dewey, “Desirable learning experiences had to meet certain stringent criteria.  They had to be democratic and humane, they had to be growth enhancing, they had to arouse curiosity and strengthen initiative, and they had to enable the individual to create meaning.”

Understanding what learning is must be fundamental to curriculum work.  To foster and encourage students to be active participants, to create meaningful learning experiences, this takes time.  It would be easier and faster to simply “pour” the information into the learners’ brains, but since that is not an option, experiences must be created and presented with enough time for students to construct new knowledge.  Therefore, items to be taught must be carefully considered since there is limited amount of time.

Cognitive Science

Today, with the help of neuroscience, educators can access neuroscience to understand how the brain learns.  Prior to Patricia Wolf’s book, Brain Matters, David Souza brought us a book called, “How the Brain Learns”: one of my teaching bibles.  By unlocking the secrets of cognitive science, it allows teachers to be strategic in creating the activities and experiences from which children can learn.  Eric Jensen is another educator and science junkie who has helped translate the current findings into effective instructional strategies. 

“The best educators will be able to integrate the current findings in neuroscience, education, and psychology to inform their instruction.” 

The middle and high school learner’s brains:

Brain development is still in process until approximately age 25.  Understanding the unique aspects of the adolescent brain is critical for educators who work with adolescents and teenagers.  As a result, curriculum and instructional strategies must take into consideration the distinctions as units of study are planned. 

It is well-noted that middle school standardized test scores seem flat as there appears to be a dip in learning.  For decades, it was believed that the brain was dormant during the tumultuous pre-teen and teen-age years.  What we have discovered is this simply is not true.  In fact, the brain is going through such a transformation, so much is actively occurring, it’s almost an overwhelming phenomenon.  Any parent of a middle school child will attest to learning, how the conversations at home have changed, how their child applies the new knowledge from their science class for example, the questions they ask, and of course, the debates they like to engage in.

This means that the adolescents in our classrooms may not have the brain maturity we assume they do.  Between immature pre-frontal cortex (part of brain responsible for planning), the synaptic pruning, the addition of hormones and the impact on emotionality and sleep, as well as other brain changes, it’s critical for curriculum developers to understand how these factors impact the learning process of adolescents.

Curriculum: Why Do We Teach What We Teach

 I have spent countless hours thinking about curriculum and discussing it with anyone who was willing to listen.  Recently I had a friendly debate with a non-educator who expressed their views that not everyone needs to go to college and that the problem is that too many people go.  I quickly put together some research about the income disparity between college educated and non-college educated and tried to explain that, for those that want to and can go, they should be encouraged to as that can provide social mobility that otherwise may not present itself. That brought me to another point.  Is the overall goal of schooling-k-12 and above, to be preparation for employment?  I argued that a well-rounded liberal arts program is worthwhile.  Even if you never “need” the course content on sociology in your actual day-to-day work life, it would be valuable to have been exposed to the content. Then there are the times when my 5th grade son asks why again he has to write another story.  “I am not going to be a writer, mom,” he says. My answer, “it’s good to stretch your brain and practice writing techniques.  You also get to share your creativity.”  I am not always sure I buy into this but the answer works for the moment. 

 Schools are challenged to educate by providing meaningful, relevant content to the masses. 

It is impossible to write curriculum unless you have a clear-cut concept of the purpose.  If you don’t know where you are going, any way will get you there.  So knowing where you are going is a critical component of any curriculum design. 

What does it mean to be educated? How is being educated different from being trained? Are both important?  What does it mean to be knowledgeable? And what should we be knowledgeable about?

Working in the schools for over 20 years, I have occasionally wondered why this, of all things to know, have we deemed so important to spend weeks teaching about. The truth is we can put whatever goals and objectives we’d like into a curriculum.  It does not mean that the goals are achievable, feasible, appropriate, necessary or even important. That is why it’s so tricky. 

And that’s just deciding the “what to teach.”  We haven’t even begun to consider the “how to teach” (in terms of organizing the content, the classroom, the strategies, the grouping, the scope, the sequence, etc.) This is no easy task.

 I was trained extensively in cooperative learning and utilized the concept as the framework for my classroom structure.  I also worked on the writing and design team to redo the middle school social studies curriculum. I was privileged enough to get to work directly with the legendary Grant Wiggins.  I worked with a team to write curriculum using Backward Design.  That experience forever altered my thinking about what I was teaching and why.  I was also trained in Problem Based Learning and developed many units where this strategy was utilized.  

For many of my students, it appears that getting through k-12 schooling will most likely be their most difficult challenge.  Once they get into the world, they can find their niche and be successful. Schools feel outdated for these students in particular.  They are skilled in so many areas not valued in current school curriculum. The technological changes and demands in our world are happening so quickly.  It is crystal clear that schools are not keeping up.

Still, after all my pondering and questioning and dissecting, I still have no clear conclusions.

As Dewey says, “education is to develop the mind not shape or mold.”

I want it all.  I want my students to be educated. I want them to know psychology, and philosophy, biology and chemistry (and physics) I want them to learn algebra and geometry.  To read and write.  To learn civics.  To solve complex problems. To think critically. To be creative and empathetic. And to be kind and open minded. Is that too much to ask a public school k-12 education for?

How Young Children Learn: Observing My Own Children Learn and Grow

 Story 1

When my daughter was 2 ½, she asked that we take all of the blue crayons from our new 96 Crayola pack out of the box.  She then asked that we color little blue swatches on blank paper, and she asked me to name each color as we examined each swatch.  I had no idea the reason why.  This activity continued for about 3 days.  A few weeks later, she was playing outside in the early evening.  I called for her and told her that it was time to come in and get ready for bed (sleeping was her least favorite activity).  She told me that it was not time to come in or go to sleep yet because the sky had not turned cerulean.  Cerulean? I was at first taken back.  What was she talking about?  I looked up at the sky and she prompted me to wait a bit and watch.  She assured me that the sky was going to change a variety of colors before it finally turned cerulean.  Then, as she noted, it would soon be dark and in fact, time for bed.  She was a creative little girl with an artistic flair.  And she was curious about the various colors of the sky.  Who knew?  And learning the names of the shades of blue helped her to solve a time problem and perhaps manipulate a bit.  I have never forgotten that story.  I tell it often for many reasons.  Usually, to remind people to take notice of the small details of our natural world.  In this case, I tell the story to remind us how children learn, how everyone learns.  Learning is a natural process that comes from observation and desire to know. 

 

Story 2

When my daughter was little we purchased several cardboard books, the durable kind that are child friendly so little hands could learn to turn pages.  By the time my son came along, all books were still in good shape.  I had been warned that having a boy was different.  He would be more rambunctious, more active, even maybe more destructive.  He wasn’t any of those things but I remember distinctly being disappointed one day to find some of the books had been torn at the binding.  After all, without instruction, my daughter had learned the purpose of the book, to look at the content.  She learned how to turn the pages in the right order and would sit and pretend to read just as I had modeled for her hundreds of times. So why was the boy destroying the books?  I had taught him the same; didn’t I?  So I watched him one day intently.  What I discovered was that my son although still interested in the content had become curious about the construction of the book.  He was trying to understand how the book was together.  Part of it was secured and closed.  But the other side was open and could move.  He was so intrigued by this simple concept.  And here it had never occurred to me that the construction of a simple object like a book could be so interesting. I am not sure what exactly he learned about the construction of the book through his discovery activity of tearing it apart, but he was curious, mystified and wanted to understand, wanted to learn, wanted to know. By watching young children explore their world, we can see the natural learning, sometimes incidental, sometimes purposeful, always meaningful.

 

We, adults, have much to learn ourselves.

 

Something to Keep in Mind When Testing Our Students

 Some thoughts about tests:

         A standard, simple but effective way to measure learning is the basic pre-test, presentation of unit (teaching of skills and content) and post-test.  Consider this example.  The teacher is doing a unit on the US Constitution.  She first gives a pre-test.  Suzy knows very little and she scores a 10%. John is a good student and scores a 77% on the pre-test.  John’s parents have taken him to Constitution Hall in Philadelphia already. They watch history channel shows with him and often discuss topics in American history. At the end of the unit, John scores a 95%.  He has earned an A grade.  Suzy’s parents do not know she is studying the US Constitution.  Even if they did, they are not able to answer her questions or provide background or additional experiences that could enhance her knowledge.  Suzy scores a 64% on the post-test.  She has earned an F.  But Suzy has increased her knowledge by 54%.  Clearly she has learned much.  John technically only increased his learning by 18%.  He is clearly proficient but he already started out that way.

What does this mean? I am not sure exactly but it’s a good question to ask ourselves. When we look at each child’s learning, we need to consider all the factors.  Suzy did learn.  And since that’s the business we are in, that learning should be celebrated for her sake and for ours.