Building Public AI Infrastructure: Four Observations from an Early Experiment in Career Navigation
Artificial intelligence is reshaping how people explore education, careers, and work. New tools promise more personalized guidance at unprecedented scale. At the same time, the field is still discovering what high-quality AI-enabled career navigation looks like and how to help those tools improve over time.
No single organization or project will answer those questions. Progress will come through many experiments, shared evidence, and a willingness to learn together.
Over the past two years, our organizations collaborated on CareerNet, an open, benchmark dataset of real career questions and answers designed to evaluate AI-generated career guidance. We view CareerNet as one contribution to a much broader effort to strengthen the field. Along the way, the project surfaced four observations that may help funders, practitioners, researchers, and technology developers working to build the next generation of career navigation.
Observation 1: Practitioners build stronger public infrastructure.
We began with a vision for an open benchmark dataset that could improve AI-generated career guidance. As we worked alongside career navigation organizations, that vision became sharper and more practical.
Our partners brought perspectives that no dataset or technical roadmap could provide. They challenged assumptions, highlighted implementation realities, and helped us better understand how AI fits into the day-to-day work of supporting learners and job seekers. Their feedback influenced both the questions we asked and the way we approached the work.
For example, partners shared the most common kinds of information that career navigators are seeking: questions like "What does a mechanical engineer do?", "How do I build a portfolio for graphic design?", and "Is community college or a 4-year university better for computer science?"
This type of input allowed the team to organize the data into seven categories and tag the data with relevant goals like “Explore Options” and “Navigate Life Constraints.” Partners specifically suggested “Understanding Purpose” as its own category after noting how often users ask questions like "What are some jobs I can realistically get that meet my passion for X?", distinct from simply exploring career options in the abstract.
That experience reinforced a simple idea: practitioners should help shape shared infrastructure from the beginning. Organizations working closest to learners often see emerging needs before the rest of the field does. Their experience helps ensure that public infrastructure reflects real-world challenges rather than theoretical ones.
For philanthropy, this means treating practitioners as co-creators, not simply future adopters. It also takes more work: landing co-creators requires additional time and effort, and grants need to be structured in ways to allow money to flow easily to various partners. But the strongest public goods emerge through partnership with the people who will ultimately use them.
Observation 2: Better guidance starts with better context.
Recent advances in AI have improved the quality of career guidance dramatically. At the same time, our work highlighted an important limitation. A technically correct answer does not always become useful guidance.
Career decisions depend on context. A high school student exploring healthcare careers needs different guidance than a working parent changing industries. Geography, local labor markets, prior experience, financial realities, and personal goals all influence what a good recommendation looks like.
For instance, prospects for data engineers might be greatly increasing in the Bay Area of California but prospects might be poor or non-existent in rural North Dakota. Similarly, a parent changing industries will need advice not only on securing a new job but also on additional classes or certifications required, costs associated with those, child care options, and other factors that might make the job transition more complex. Ultimately, this contextual information must be taken into account to determine whether an LLM responds appropriately and comprehensively to the unique user.
As we evaluated AI-generated career guidance, we found that context often determined whether an answer felt actionable and relevant. To address this, we included tags that indicate the underlying goal the asker is trying to achieve, as described above. For example, "Should I go to college right after high school or take a gap year?" was labeled Take Action, since the asker isn't just asking about college versus a gap year in the abstract, they're trying to make a decision. Future public infrastructure should help AI systems understand the circumstances surrounding a career decision as well as the question itself in order to answer it well.
The next generation of career navigation will benefit from richer context, stronger labor market intelligence, and better ways to evaluate whether guidance actually fits the individual receiving it.
Observation 3: Building a public good is easier than integrating one.
We assumed that organizations would adopt a high-quality, open benchmark if it addressed an important need. Instead, we found that adoption depended on something much more fundamental: whether organizations could realistically incorporate it into the systems they had already built.
Many career navigation organizations had invested significant time creating their own AI workflows, data pipelines, and technical infrastructure. For instance, our partner CareerVillage has an AI chatbot called Coach. Another partner SkillUp will be testing a new sub-agent this fall. Integrating a new benchmark or dataset might require reworking those systems, testing new approaches, and dedicating engineering resources that many organizations simply do not have. Interest was high. Implementation proved much harder since it required more than simply fine tuning a model.
That experience raised broader questions for the field. As AI capabilities continue to accelerate, how do we help mission-driven organizations keep pace? Which organizations will have the technical capacity to adopt new tools and shared resources? What partnerships, technical assistance, or new approaches will help more organizations benefit from advances in AI?
Those questions extend well beyond CareerNet. They will shape whether the next generation of AI-enabled public infrastructure reaches the organizations and communities it aims to serve.
Observation 4: Shared learning accelerates progress.
Every project generates insights. Some improve technical design. Others reveal implementation challenges or identify emerging user needs. Those lessons become far more valuable when they inform work beyond a single organization or grant.
Throughout this project, conversations with partners like CareerVillage and SkillUp strengthened the resource, and the work itself generated new questions for those same partners. For example, partners wanted to easily update the data since labor market information can shift quickly and vary widely by location: median wages, job openings, and growth projections for a registered nurse in Topeka look very different from the same role in New York City.
This discussion led us to incorporate tags on the dataset so data for specific areas could be refreshed as updates are available. Practically speaking this means that as an LLM responds to a user question about their career path and potential earnings in Topeka, it will consider the relevant labor data that may impact their decision and generate an answer distinct from what it might say to a user asking the same question in New York City.
The field can build on that approach. Funders, practitioners, researchers, and technology developers each see different parts of the landscape. When they openly share evidence, compare approaches, publish what they learn, and contribute to common resources, everyone moves faster. That includes sharing implementation challenges and unexpected findings alongside successes.
AI will continue to evolve. Our ability to learn together should evolve just as quickly.
Looking Ahead
CareerNet represents one early experiment in a rapidly changing field. It answered some questions and raised many others. We expect future efforts to improve on this work, challenge some of its assumptions, such as the consistency and relevance of SOC codes, and contribute new ideas that strengthen the ecosystem.
The opportunity extends beyond building better AI tools. It includes building the shared infrastructure that helps those tools improve over time. It includes creating stronger connections among practitioners, researchers, technology developers, and funders. And it includes treating knowledge itself as a public good that grows more valuable each time someone builds on it.
We hope these observations contribute to that ongoing conversation.
Marie Groark is Managing Director of the Schultz Family Foundation.
Kumar Garg is President of Renaissance Philanthropy.
Meg Benner is President of The Learning Agency.