The Knight Foundation News Challenge, Open Source, and the Future of Hyperlocal

(Quick Update 10/11/09 – see Zachary Seeward’s post about how the Knight Foundation is considering changing the terms of grants in the future, as well as Patrick Thornton’s piece on how the Foundation is assembling a team to continue working on the code base produced by the Everyblock team).

The John S. and James L. Knight Foundation, among many other philanthropic initiatives in culture, community, and journalism generally, has been running the Knight News Challenge since 2007. Its basically a grant competition, in which various digital journalism initiatives compete for a pool of grants amounting to $25 million total over five years.

One aspect which makes the Knight News Challenge unique – other than the size of the grant pool – is that the winning grantees are required to:

1. Use digital, open-source technology.
2. Distribute news in the public interest.
3. Test your project in a local community.

It looks like a fantastic strategy: encourage innovation, provide funding without forcing the grantees into short-term, must-build-immediate-ROI type thinking, and share the results with the broader community through open source.

Knight - Photo by Ruth L., cc-by-nd license
Knight - Photo by Ruth L., cc-by-nd license

Two recent successful projects from Knight Foundation grantees – EveryBlock and Village Soup (which I’ve written about before in this blog), however, suggest there might be some gaps in the Foundation’s overall plan.

The core of the issue is this question: once the Knight Foundation funding is expended, what happens to the open source project the grant process mandates?

Do the creators truly create, engage with, and sustain an open source community around the code they release, contributing to and supporting the open source version, or do they “take it private”, leaving the open source seed to either take root and grow (or wither) on its own?

First, Village Soup. When I wrote about them back in May, it was unclear what exactly would be released and in fact whether or not they were compliant with the terms of the grant:

As one of the commentators on [founder Richard M.] Anderson’s recent blog entry on making hyperlocal pay pointed out, however, that doesn’t seem likely to be what the Knight Foundation expected when it funded creation of an open source project. Perhaps we’ll hear more as the end of the grant period (June 2009) approaches?

Anderson himself later commented on that same post, clarifying:

In accordance with the terms of our Knight Foundation News Challenge Grant, we are using the funds to create an open source version of VillageSoup’s software, which combines blogs, citizen journalism, online advertising and reverse publishing from online to print. The Knight Foundation will sublicense the open source publishing system software to third parties under the GPL and Creative Commons License.

In fact, that code is now available from this google code project: vsce (Village Soup Community Edition). It’s GPL (v2) licensed, with content (the user manual?) available under a Creative Commons Attribution 3.0 license.

There’s a 1.0.0 release, dated July 15, 2009, as well as a 63-page user manual which covers installation, configuration, and operation of sites based on the platform. The platform components are pretty standard in the open source world: Java (specifically Java Server Faces), JBoss Application Server, JBoss Seam, Hibernate, MySQL, Maven, and JQuery.

Fighing Knights - Photo by threlkelded, cc-by-nd license
Fighing Knights - Photo by threlkelded, cc-by-nd license

What’s missing, though, is any real sense of an open source community around the platform. Issues? none. Wiki pages? none. There’s only one check-in to the subversion code repository, with no changes since then. The only “person” attached to the project at Google Code, and also the project owner, is identified as “helpd…” (no great imagination necessary to suggest that this parses to

In other words, an open source project has been released, to comply with the terms of the Knight Foundation grant, but is it an open source project likely to succeed?

There’s nothing in GPL v2 nor in the Knight grant itself, so far as I can tell, that would prevent (or even, for that matter, strongly discourage) VillageSoup® from continuing to iterate on, improve, develop, and maintain the Enterprise (hosted) version, Village Soup Common, without contributing those fixes to the open source community edition, and simply let the VSCE project wither on the vine.

EveryBlock, another Knight Foundation News Challenge grant recipient, has received considerably more press coverage (the acquisition of EveryBlock by MSNBC this week was covered by the AP and the NYT) but is potentially in a similar situation: there is an open source project release, in accordance with the terms of the grant, but will it be sustained long term?

A few pointers to other blog posts laying out some of the issues:

The emerging consensus seems to be that EveryBlock has fulfilled its obligation to the Knight Foundation, releasing the project under the GPL (v3 in their case) at the end of the grant period. It also seems clear that neither the GPL itself nor the Knight Foundation grant will require that MSNBC continue to make its changes to the project available as open source.

So MSNBC could continue to improve the code, starting from the GPL code base, without releasing those improvements to the open source project. Since whatever platform they offer will almost certainly be a web service, they will not be distributing the modifications and can keep them private.

Whoa! You Totally Conquered Him, Dude! Photo by Sister72, cc-by license
Whoa! You Totally Conquered Him, Dude! Photo by Sister72, cc-by license

But here’s the rub: so can anyone else. I could also take the GPL’d EveryBlock platform, improve upon it by adding additional features, and run a hyperlocal site for Salem MA, without being obligated to redistribute my version. That’s how the GPL works.

What, precisely, did MSBC buy, then? Presumably, the people involved in the project, the name and domain, and perhaps the existing data (it isn’t clear to me what license the data itself is under).

Will MSNBC take EveryBlock private, or will they learn to value the benefit of working with an open source project, and sustain a real community around the codebase?

I was initially more optimistic about EveryBlock, since the ebcode site has some activity: code updates, individuals named as owners, a mailing list). ebcode is also built on top of Django and Python, which will connect them more clearly to other communities of open source developers working on journalism. But in the paidContent interview EveryBlock founder Adrian Holovaty reportedly said:

others will only have access to the code as it existed on June 30—when it was initially released—meaning will likely have an edge over any competitors. “What happens after that we’re not obligated to make that open source,” Holovaty says, adding that so far only a handful of sites have actually adopted the code.

He goes on to say, in the same interview:

The Knight Foundation funded EveryBlock via its Knight News Challenge program but the foundation did not have equity in the startup. “Basically the grant was paying for development of the open source code and we fulfilled the obligation,” Holovaty says. Asked whether he was now considering returning some money to the group, Holovaty says he is “planning on pointing everyone I know to the News Challenge. That’s what they’ve asked me to do.”

This doesn’t bode well for the long term future of an open source project derived from that code.

Ritterrüstung - Photo by marfis75, cc-by-sa license
Ritterrüstung - Photo by marfis75, cc-by-sa license

What does this trend (if two can be called a trend) begin to suggest a flaw in the News Challenge approach?

If the grantees are required to release an open source version of the code written during the grant period (and maybe only the subset of the code specifically funded by the grant money), but have no real investment in the community model, and no real open source community of contributors around that core, is there any real benefit?

One could argue that if these platforms prove valuable enough, the GPL’d core that comes out of the grant period could be taken by a community and “forked” to create a real vibrant open source project around them – but generally code dumps (significant sets of code that were created by others and then thrown over the wall into an open source community) lead to less successful open source projects than those which actually develop organically from the beginning. It’s difficult to find a group of developers interested in making a community around existing source code – you’re more likely to find a community of developers willing to contribute to creating a code base.

Put differently, communities are great at creating (and maintaining, supporting, extending) code: code is not great at creating communities.

Should the Knight Foundation, and the News Challenge in particular, be doing something else to encourage or require real communities to form around the open source projects? It’s difficult, even with the purest intentions, to ensure that a real community will evolve around any open source project – though getting the community involved throughout might go a long way in that direction. How would open source developers contributing to the effort but not partaking in the grant funding feel?

On the other hand, is this a case where everything is actually working as it should be? If communities evolve around the open source projects that’s great, and if they don’t then perhaps there was no need to release the open source version, but there was no harm in doing so either.


  1. Pardon the “comment-on-your-own-post” but one quick update:

    Via a comment by Michael Bernstein on this O’Reilly Radar blog post, I was pointed to this thread on mailing list for the ebcode project, in which Adrian Holovaty says:

    . . . our grant has run out, and we’re no longer
    obligated to open-source anything from this day forward.

    But, with that said, we’re going to see what sort of usage the code
    gets and play it by ear. It’ll be interesting to see what happens,
    given it’s an extremely niche product, as opposed to a general-purpose
    framework like Django.

  2. Pingback: kdavis (KDavis)
  3. I’m not sure why you struck through the text that Everyblock was built on Django. It most certainly is built on Django.

    I think you are right that the future isn’t bright for Everyblock developing in to a vital open source project with a strong community if the original developers aren’t initially working to foster that community.

    The news challenge seems to be designed to give people a chance to explore and prove an idea. In exchange they are expected to share what they learned.

    You can argue that developing a community around their code should be part of the requirement to share what they’ve learned, but developing community is a significant amount of work in and of itself and so would take away from the time and effort available to explore the prove the idea in the first place. The source code, on the other hand, is an efficient expression of at least some of the learning they’ve done, it is a necessary part of proving the idea in the first place.

    I think the news challenge strikes the right balance here. Whether or not a community forms around it, they’ve released some interesting code for harvesting public data, and for presenting it. I think in the next week or so I’m going to try getting it running on one of our servers so we can explore integrating some of the information it acquires into a project we are working on. We’ll see what happens after that.

  4. Pingback: Rams (S.Ramaswamy)
  5. Just a wee observation: the “Use digital, open-source technology.” requirement wasn’t part of the rules in 2007.

    And, while Knight requires grantees to publish our source code using an open source license (I think they specify only “GPL”) they have never (to my knowledge) asked people to build any kind of community around the software they’re developing. Community using the news? Sure. But not community building the infrastructure.

    I suspect that Knight mostly got what they wanted out of this–even though the project got bought up, the code is still there for someone else to pick up and build from. Similarly, had Everyblock folded post-funding the code would still be there for anyone who didn’t want to see it die.

  6. Thought I had read somewhere that it was not using Django, which is why I struck it – I’ll go back to the original – thanks.

    I guess the question comes down to intent – what the Knight Foundation hopes to get by requiring a GPL release. If what they require is that the code they funded be available for use by others, they have succeeded. If their intent is to create a viable community and associated open source project, perhaps they should reconsider how the requirement is stated.

  7. Thanks Amanda – didn’t realize that requirement was added later.

    I agree this may be “functions as designed” – and those who are disappointed may by missing the point.

    But what if the Knight Foundation required an Affero GPL license? That also would’t auto-magically create a vibrant community, but it would mean that would be obligated to continue to make code available for their improvements and derivative works.

    On the other hand, that likely would have prevented from doing the acquisition at all, and if there was no community in place to pick up, support, and continue to drive development, the project might have ended there.

    I’m not at all convinced that this would be a better outcome for the Knight Foundation, for the EveryBlock folks, or for the community at large.

  8. … it would mean that would be obligated to continue to make code available for their improvements and derivative works.

    I’m not sure you’re right about that — owns the copyright now. They have privileges that the rest of us don’t if they own the copyright.

    Which isn’t to take a position on what is right or wrong, just to say that in the present situation, had the code been released Affero-GPL, would still be able to take it and run with it.

  9. Just FYI, MSNBC isn’t buying EveryBlock, but is.

    They’re half-sisters. MSNBC is the cable TV company, owned by NBC, based in New York, while is the news Web site, owned by Microsoft and NBC, based in Redmond, Wash. It’s the Web site that’s buying EveryBlock.

  10. True, I hadn’t thought about that fact. Since the copyright ownership is complete (no outside contributors) can use the code as it likes, and doesn’t have to obtain it under and AGPL license. (Just because it was released to others under such a license).

    This is more like commercial open source companies who make code available via the GPL or other open source licenses, but can also make it available under a commercial license, since they own copyright on the code.

  11. The KNC was/is an evolving experiment. I think it was reasonable of Knight to assume that after a year or two of support, a successful open-source project would find the right community of contributors and continue on its own.
    My experience as a KNC awardee is that the Knight Foundation is being as critical of their approach as anyone, and they are looking for ways of modifying it to address the very concerns you mention above.
    IF they haven’t already announced it, I think its likely that they will limit future KNC grants to nonprofit organizations, as an indicator of commitment to communities over profitability. I also think they (and we) are learning more about what it takes for an open-source project to become a success, and they will implement additional evaluation, documentation and other needs identified in future iterations.
    The Knight Foundation seems to me to be very adaptable. They do not presume to have all the answers, and as long as they want to learn from any mistakes, we should all applaud that they’re willing to make some.

  12. The primary reason for requiring that the source code of funded inventions be published is to prevent a grantee from establishing a monopoly. If a recipient develops a breakthrough technology, they’re denied monopoly status in the marketplace for their idea. Thus, the foundation fulfills its altruistic purpose of developing innovation for the public good. Meanwhile, the funded source code faces a Darwinistic outcome of success or failure determined by its merit.

Comments are closed.