In the summer of 2008 I was happily working as a professor at Syracuse University when I received an unexpected phone call. For the previous year the American Enterprise Institute—one of the oldest and best-known think tanks in the country, where I had a part-time affiliation—had been searching for a new president. Was I willing to be considered for the job?
The think-tank industry is very small, so it has no established pipeline for leadership. Boards are never exactly sure what type of people should lead these organizations, and executive searches frequently prove challenging. I happen to know I wasn’t the first choice—or the second or third. For years I’d taught and written about fundraising and managing nonprofits, but I’d never actually practiced either. I’m convinced that the last thing AEI’s directors said before offering me the job must have been “Ah, what the hell—let’s give him a shot.” But I got lucky. They made the offer. I accepted.
Working as a first-time chief executive is a challenging proposition on its face. But I faced more than just the standard learning curve: Between the time I accepted the position, in mid-2008, and when I started, the following January, the nonprofit economy imploded as a result of the Great Recession. Charitable giving in inflation-adjusted terms fell by almost 10% from 2008 to 2009, most of it during the last quarter of the year. AEI is fully dependent on charitable giving, accepting no government grants or contract research; thus its revenue plummeted just as I was walking in the door. The institute had to compete for every dollar as never before. My new colleagues and I had to show donors why their shrinking philanthropic investments should come to us rather than go to others, and how investment in our work would create tangible impact.
Even before the recession, demonstrating impact was a growing concern in the nonprofit sector. For-profit businesses can quantify their impact using metrics such as sales and shareholder returns, but nonprofits can’t. That’s doubly true for nonprofits in the world of pure ideas, like AEI. What is the metric for us—clever thoughts per hour? Yet modern philanthropists, many of whom have made vast fortunes in analytics-driven high-tech businesses, demand proof that their charitable dollars are doing real good. Mark Zuckerberg and others like him won’t give to organizations that swallow up their money without producing measurable results. His generation insists on evidence that they’re creating value with what they give. They need to see data. Having an intangible product is no excuse.
Show impact or fail—that was my initial major challenge as president of AEI. It led to plenty of sleepless nights in my first couple of years on the job. But meeting that challenge turned out to be more than a question of my personal survival in a new role. It has fundamentally changed how we do business.
A Golden Opportunity
To leave a tenured job for an uncertain future as a nonprofit manager may sound imprudent. It was actually of a piece with the behavioral patterns I had established up to that point in life.
I grew up in Seattle in a family of artists and academics. From a young age, I showed an aptitude for music, and I quickly found my way to the French horn. I gave college a shot after high school but left after a year and set out with my horn to make a living as a classical and jazz musician in the United States and Europe.
A few years into my career as a musician, I met a girl from Barcelona and took a job with an orchestra in Spain in an attempt to persuade her to marry me. We had no common language when we met, but having recently celebrated our 26th anniversary, with three kids nearing adulthood, I’m pleased to report that our communications skills have improved since that time.
I loved being a musician, but it always bothered me that I’d never finished school, and that nagging sense only grew over time. So while still living in Spain, I restarted college via correspondence courses.
A month before my 30th birthday, I walked out to my mailbox and collected my bachelor’s degree in economics. Not long after, I walked away from music entirely and returned to the U.S. to pursue a master’s in economics, which led to a PhD in public policy analysis, which led to work as a university professor. I eventually landed at Syracuse.
During my decade in academia, I taught a lot of students from administration and MBA programs, and much of my research focused on nonprofit management and social entrepreneurship. I wrote a textbook on the subject. One of the central questions I kept confronting was that of value creation: How do nonprofits, except those that deliver the most tangible products or services, know they’re creating real value? The field, myself included, never seemed to have satisfactory answers. I’d talk a lot about “social return on investment” and “the double bottom line,” but those were more theoretical concepts than practical, actionable guidelines. The move to AEI struck me as a golden opportunity to sort out these theoretical puzzles from within the real world of management.
But the decision to move to AEI was also deeply personal: I was a true believer in the organization’s core principles. AEI is strictly nonpartisan, eschewing political entanglements and institutional positions. What unites the scholars and the staff is a shared commitment to a simple moral maxim: The free-enterprise system and American leadership in the world are pillars in the fight to defend human dignity against poverty and tyranny and to help lift up people on the margins. Learning about the amazing antipoverty record of global capitalism and free trade was a huge part of what had attracted me to the study of economics in the first place. Now I had the opportunity to dedicate all my work to sharing this truth with the world.
My deep admiration for AEI actually posed a bit of a problem. What if, as an inexperienced executive, I failed, hurting the organization in the process? When I was trying to decide whether to take the position, I had breakfast with Tully Friedman, the private equity pioneer and a longtime AEI board member who later became AEI’s chairman and one of my closest friends. I worried aloud about what would happen if I failed in the job and imperiled this 75-year-old think tank. “Don’t worry about that,” he said. “If you can’t raise money or motivate the scholars, we’ll fire you within the year, and AEI will be just fine.” This was oddly reassuring. So with a mix of excitement and panic, I tossed my tenure and made the move to Washington.
Not Outputs or Inputs but Impact
Nonprofit leaders typically make two mistakes when trying to answer the question “How do we know if we’re having an impact?” The first is what I call the sui generis error. It’s the idea that our work is unique and our organization is so unlike anything else that exists that we can’t compare it with—or measure it against—any other organization. It’s amazing how many smart people really believe this about their own organizations or those they love. It’s a spurious claim, but I hear it all the time. And philanthropists hear it even more often.
What is the metric for nonprofits in the world of pure ideas—clever thoughts per hour?
The second mistake is the “lamppost error,” named for the story about a guy who loses his keys in the street and spends hours looking for them under a lamppost because the light is better there. Nonprofits struggling to measure effectiveness will frequently turn to whatever is easiest to see—usually inputs such as how much they’ve received in contributions or outputs such as how busy they have been. This is obviously inadequate, because what we’re really interested in isn’t inputs or outputs but impact.
The sui generis error leads to not measuring anything, and the lamppost error leads to measuring the wrong things. My colleagues and I set out to avoid both pitfalls and create a better way to understand and describe our true progress.
AEI’s output is pretty straightforward: books, research articles, op-eds, media appearances, public events, and so on. These products effectively constitute our supply curve. But nobody contends that simply writing an op-ed, publishing a peer-reviewed paper, or booking a scholar on television automatically guarantees a change in how leaders think and act. As a result, our output metrics are not particularly interesting in isolation. To move from output metrics to impact metrics, we had to overlay a demand curve on our supply curve. We had to find ways—even imperfect ways—of measuring how much leaders wanted and sought our work.
In the ideas industry this kind of demand is almost never directly observable. No matter how much excellent work a single organization may be doing to educate leaders or raise the profile of a given issue, no metric such as public opinion polls, election results, or legislative votes can show the organization’s role amid the static of thousands of other variables.
We had to find ways of measuring how much leaders wanted and sought our work.
Instead of direct impact measures, we realized, think tanks need to develop proxy impact measures, by searching for where their supply curve intersects with competitive demand in the marketplace of ideas. We had to identify and track the products that leaders consume at some cost to themselves and in a direct trade-off with alternatives. In isolation, each of these metrics is a single point of limited utility. But considered together, they paint a pointillistic picture that helps us assess the impact of our work.
Here are just two examples from the suite of proxy metrics we’ve come to use:
The most prestigious national newspapers each receive about 1,000 unsolicited op-ed submissions every week. The “market” for real estate on those papers’ opinion pages is unbelievably competitive, and editorial staffers ruthlessly reject everything except what they believe readers most need and want. That selectivity can reveal competitive demand for our product. Thus, although total op-eds written per year is just an output metric, the number of op-eds our scholars publish in a defined set of the most competitive outlets constitutes a viable proxy measure for impact. Though far from a catchall figure, this metric has one particularly useful aspect: The data is by definition public and thus allows comparison with peers and competitors. Every year to date, AEI has maintained a lead.
We do the same thing with congressional testimony. Most policy experts want to testify on the Hill, but they can’t just call up the Senate switchboard and get on the docket. The Senate calls you. So although nobody believes that public testimonies are the only way think tanks shape the debate, they do offer one useful angle for measuring impact. And again, it is helpful that everyone’s data is a matter of public record. When we initially analyzed these numbers, during the 110th Congress (2007–2009), AEI was in fourth place among think tanks in terms of congressional testimony. By the 111th Congress we were number one, and we’ve maintained that position ever since.
It’s worth emphasizing again: Neither of these metrics is a direct measure of impact, let alone a perfect one. All such proxy metrics can be gamed or misunderstood, and each reveals only a partial dimension of the truth. And not all worthwhile proxy metrics can be used to compare competitors. For example, our scholars’ personal relationships and private briefings with policy makers and journalists are prime proxy metrics of impact, because the leaders’ time and attention is so scarce. But it is impossible to compare such proprietary data across organizations.
The point is not to search for one perfect proxy. Instead, nonprofits operating in the ideas industry can build a dashboard, outfit it with a wide variety of variables like these, and then use it to gauge the revealed preferences of public leaders and the uptake of the organization’s work.
Clarifying the Mission
At the same time we began constructing this impact dashboard, my AEI management colleagues and I made several complementary changes. For example, we reworked how the organization talks about itself to better reflect our definition of success. Think-tank mission statements are more often uninspiring product lists (“We produce high-quality policy research”) than true expressions of purpose. In consultation with our scholars and supporters, we wrote a new statement of purpose: “The American Enterprise Institute is a public policy think tank dedicated to defending human dignity, expanding human potential, and building a freer and safer world.
The work of our scholars and staff advances ideas rooted in our belief in democracy, free enterprise, American strength and global leadership, solidarity with those at the periphery of our society, and a pluralistic, entrepreneurial culture.” We tried to make clear the moral goal of our ideas, which is to serve others, especially those at the margins of society.
Measuring Impact
Op-Eds
Think-tank scholars write many op-eds, but that’s a measure of output. To better understand its impact, AEI began tracking how many of its op-eds were published by three leading newspapers—the New York Times, the Wall Street Journal, and the Washington Post—and how its placements compared with those of competing think tanks over time. The percentages reflect yearly averages for the period 2015 to 2017.
Congressional Testimonies
Testifying before congressional committees is another way AEI scholars can have real influence. When Arthur Brooks arrived and urged AEI to start gathering this data, the organization ranked fourth on this measure. Since then it has moved into the top slot, and its market share has grown.
We then launched a series of high-profile entrepreneurial ventures expressly designed to create impact and fulfill our new statement of purpose. Some of them are quite unusual, such as research on happiness, experimental multimedia ventures, and a major collaboration with the Dalai Lama. Our newest venture in this series focuses on despair and human dignity in America. Its aim is to find and propose long-term solutions to the underlying causes of the social and economic despair that defined both sides in the most recent presidential election. The program has four main pillars: vocational and technical training to help people find good jobs; using free enterprise to strengthen the economic security of families; reforming the criminal justice system; and finding new strategies to curb the epidemic of opioid abuse.
Built and marketed like an internal start-up, this venture does not displace any of our other work. To promote it to investors, we created a prospectus and a pitch deck, just as any start-up does. We approach would-be donors as we would venture capitalists (which many of them are). We talk about the ROI and establish the basket of impact metrics that we will use to demonstrate success.
Our new focus on impact has also helped us refine our audience development and segmentation. We believe that in the marketplace of ideas, we can classify potential consumers of our products into four groups according to their receptivity to a given message: true believers (who already agree), persuadables (who are open to hearing from us), hostiles (who think our perspective is stupid or evil), and apathetics (who couldn’t care less). We cross these four attitudinal groups with the five key demographic groups—policy makers, business leaders, the media, community leaders, and academics—that form our target audience. This gives us a 4 x 5 matrix and enables us to balance our strategies and offerings across these groups to maximize impact and more effectively serve our mission principles.
Our Strategy Paid Off
As you might expect, these changes weren’t always easy to implement. I made plenty of mistakes along the way. Some colleagues complained that I was asking them to spend too much time and energy collecting data—and in some cases they were right. More than once, I fell prey to measuring the wrong thing entirely. For example, I became concerned when attendance at a series of live events started to trend down. But after a few months, someone pointed out that we’d begun live-streaming the events on the web, where they were getting a lot of traffic. This insight led to an even better metric, subscribers to events and original video programming on our YouTube channel—a measure by which AEI now leads the think-tank industry.
Despite a few hiccups, our strategy has paid off. Since 2008 AEI’s operating revenues have averaged annual growth of roughly 10%; the institute bought and renovated a new headquarters in downtown Washington, DC; and we have grown from 140 full-time scholars and staff members to 220. More important, we’ve seen the impact of our research increase among policy makers and other leaders in dramatic ways. It’s an exciting time at AEI.
Naturally, this growth necessitated changes to our management structure. Managers now have more authority and autonomy regarding internal administration, while my role has become increasingly outward-facing. These days I’m on the road about half the time, meeting with prospective investors and delivering 175 speeches a year about our work.
As I look back on my early days leading AEI, I am grateful for the scary exigencies that led to the changes we made. Because donors demanded evidence that our inherently amorphous products were having an impact, we learned to measure and demonstrate that. Given the imperative to spread the word about our work in a crowded marketplace, we made investments in our communications systems that will pay off for years to come. All in all, I got vivid proof that crisis is the mother of invention.
Other nonprofits are now asking about our systems, and several major foundations have asked our staff to help their other grantees adopt some of our practices. It’s satisfying to see this. But most of all, I’m grateful to be able to help our fellow social enterprises increase the impact of their efforts in a world that needs them more than ever.