Groundhogs, Meteorologists, or DarkSky?

Evidence-based Strategy and Design in Education

three approaches.jpg

Suppose you need to plan your travel schedule for coming weeks around the weather. There are three approaches you might take. Each parallels a method leaders supporting education could use to inform strategy and design decisions with empirical evidence.  

Since today is Groundhog Day, you could turn first to America’s most famous rodent, Punxsutawney Phil, and embrace his predication of another six weeks of cold and snow. Groundhog Day is popular and widely known. It yields a quick, definitive prediction, nearly for free. It’s easy to create buzz, and a vocal group swears by its accuracy (even as historical data suggests it’s wrong 69% of the time). The situation gets a little more complicated, however, if you look at multiple groundhogs. While Dunkirk Dave agrees with Phil, General Beauregard Lee, Hutty the Hog, Staten Island Chuck, Fred la marmot, Wiarton Willie, and Shubenacadie Sam all predict an early Spring. Do you go with an average prediction? Does Phil’s take get more weight since he is more widely recognized than the others?  

Using Groundhog Day as the evidence with which to predict the weather is similar to relying on stories from the people you serve or brief consultations with experts to inform organizational decisions. Often, the stories are engaging and the guidance quick and clear. However, stakeholders will have different experiences and experts’ opinions often conflict, so it’s often up to you to figure out which to embrace. Just how representative these varied viewpoints are can be hard to determine. Whichever path feels right to you, there are often stories and expert opinions available to support it.  

Alternately, to inform your weather planning, you could hire a meteorologist to gather data at your locations and apply their original, award-winning, peer reviewed climate model to provide a custom prediction. A month after packing up their hygrometers, pyranometers, and disdrometers, they would hand you a thick report and a large invoice. Their predictions would carry the brand of a top-tier research institution and reflect a detailed picture of the peculiarities of where you work. Among the charts and equations, their report would make clear that while short term projections should be quite accurate, current science can’t actually do a whole lot better than the groundhogs six weeks out. 

Conventional educational evaluations look like this. They are rigorous, focused, highly credible, slow and expensive. Often, your situation and strategy may already have shifted by the time analysis is complete. Unless a funder mandates such a study, the cost is often difficult to justify given competing priorities. This is especially true because the resulting predications can be frustratingly narrow. The more conventionally rigorous the research, the longer it takes, the more it costs, and the narrower the results are likely to be. They fail to inform many of the questions that arise in strategy development and the design of products and services.  

The third weather predication option is Dark Sky. Dark Sky is an app that draws on a wide range of existing weather sensors around the world linked to multiple, scientifically validated forecasting models to provide hyper-localized weather forecasts. The results are available in real time and presented through accessible and compelling visualizations. DarkSky doesn’t even attempt to predict six weeks into the future. The ten-day forecast the DarkSky does provide, however, is likely nearly as accurate as the custom prediction. The app isn’t free, but compared with hiring your own meteorologist, the price is right.  

Our approach at the Cambridge Learning Group mirrors the strengths of the Dark Sky approach. Predicting the performance of educational products, services, and interventions is at least as complex as modeling the weather and much more difficult to standardize. We are a long way from having an app for that. However, analogous to Dark Sky’s use of existing sensors and models, we believe that our team can use the expansive body of existing research across academic and professional fields to yield evidence-based, actionable recommendations on a wide range of strategic questions, quickly and cost-effectively. 

This approach goes beyond the cursory literature reviews that often accompany conventional evaluations or the cherry-picked lists of studies that are common in marketing materials. A systematic, rigorous, multidisciplinary synthesis of research directly applicable to your mission, theory of action, business model, and operational context can both validate your current approach and provide powerful insights for innovation and improvement across functional areas. 

Realizing that research is only as valuable as the action it informs, we don’t stop at analysis. As DarkSky uses information design to make forecasts accessible and usable, so do we work with our clients to communicate the results of research synthesis is differentiated for each of their key audiences. Using our cross-functional design and management expertise, we then help clients implement whatever recommendations our analysis yields.