Introducing NEO, TPM’s New Frontpage Publishing Software

Longtime readers have witnessed the gradual evolution of TPM — from Josh Marshall’s blog to national politics site —  and are accustomed to hearing from Josh when significant changes are made. And while Josh has written a post giving some initial guidance on the new design, we thought readers might enjoy some insight on the project from the perspective of TPM’s product team.

At 6:23 PM on Tuesday night, approximately an hour before we’d planned to launch the new frontpage, a colleague sent me the following message over Slack: “This will go down as a legendary day in TPM history. Incredible timing.” Earlier in the afternoon, Michael Cohen had plead guilty in Manhattan Federal Court. A few hours later, Paul Manafort was found guilty on 8 counts. These were two stories TPM had been obsessively reporting on. It was a big news day. And while it’s typically not best practice to launch a major new product in the midst of breaking news, something about the timing did seem incredible. Moments like this are what TPM is built for.

When you make significant changes to a website, especially one with a deeply loyal readership, you owe those readers an explanation. And while we might have been brash (or foolish) to unveil our new look amidst a fusillade of breaking news, we’ve been slightly delayed in sharing the motivations and implications behind the new changes to the site. Part of the reason for the delay is intentional — we were hoping readers would experience the site and share their unfettered feedback (you have, thank you). The more immediate reason is that we are a small team and were busy putting out fires until the moment the new site went live.

Now that we’ve had a moment to catch our breath, we want to properly introduce you to the new TPM. However, before digging into what’s new or what’s changed, it might help to briefly touch on what makes TPM’s frontpage unique and explain a few of the editorial objectives it serves.    

A substantial percentage of TPM’s pageviews are from readers coming directly to our frontpage. This high level of direct traffic is the result of several factors, including our legacy as a single-page blog, but it is also by design. Unlike more general news sites which typically contain several verticals (technology, entertainment, etc.), TPM is fairly singularly focused. We cover politics, and within the sphere of politics we tend to latch onto a few key stories and cover those stories obsessively (the Manfort trial, for example). Regular readers are familiar with TPM’s style of iterative journalism where an important and ongoing story arc (what we refer to as a storyline) is covered not just by one article but through a number of articles, blog posts, or short 2-3 sentence Livewire updates. In theory, a reader should be able to visit TPM’s frontpage and get a comprehensive overview of what storylines we are covering on any given day and get a sense of how certain pieces of that storyline fit together. When understood this way, it makes sense that the majority of traffic would come to the frontpage, since the frontpage, not the article, does the bulk of the heavy lifting when it comes to conveying information to our audience.

However, over the past fews years it had become apparent that we weren’t doing a good enough job of highlighting and meaningfully connecting the key stories we were covering. We’d never really had a great method of visualizing storylines and our technology had lost step with our editorial needs. This was partly a design problem, but the more fundamental challenge was that we didn’t have the right tool.

To solve this problem we built NEO, our new frontpage management software. Without going too deep into technical details, NEO works by communicating with the WordPress API to pull in article data which it then translates into Preact.js components. From our editors’ perspective, NEO is fully integrated into the WordPress admin and editing mode can be toggled on and off through a switch on the frontpage. With edit mode enabled, the editable sections of the site come into focus and users have the option to add and remove articles, create custom frontpage headlines, swap in different images and select from one of three main page layouts:

1) The Editors’ Blog at the top left of the page with the feature package to the right

2) Livewire at the top right of the page with the feature package to the left

3) A full-width feature package with the Editor’s Blog and Livewire directly below

These are essential functions, but they existed in one form or another in the old system. Where the real advancement comes in is in the ability to create what we call storyboxes. A storybox is a curated grouping of between one and ten articles and replaces the stream of single articles that used to run down the middle of the page.

To create a storybox, an editor can manually select individual articles from an article feed or select bunches of articles using a query filter (for instance, show the latest five articles on the Russia Probe). Each storybox requires the editor to select one of five layout templates. These templates were designed to be used at an editor’s discretion for the purpose of creating visual hierarchies between articles of varying substance, timeliness, etc. New storyboxes can be added (and old ones removed) with as little effort as a few clicks. And each can be given a header label to alert the reader to the type of content it contains.

For example, a storybox containing reporting on the Manafort Trial storyline might be labeled “On The Scene at the Manafort Trial” and have a layout where the more substantively reported articles contain an image and a large headline while articles providing only a short update have a smaller headline with no image.

The upshot, from an editorial perspective, is that NEO makes it relatively easy for editors to group articles and visualize how certain stories and storylines fit together. It also allows editors to highlight original reporting and create feeds that automatically update when a new article is published. And while that might all sound great, the obvious question is what does this mean for readers?

The primary objective of the new site was to save readers time and to help them start to see story arcs less like a reader and more like a TPM editor. Where previously readers were often left to their own devices (and occasionally Josh’s Ed Blog posts) to piece together how various parts of our reporting combined to define the bigger picture, we wanted the new front page to do a considerable amount of that work for them. Rather than being forced to scan an undifferentiated stream of headlines looking for articles concerning the fate of Michael Cohen, several of which might no longer even be on the frontpage due to the space constraints our old frontpage had, a reader of the new site could expect to find all of that day’s Michael Cohen coverage in single location and have a good idea of what articles TPM editors thought were the most important. Prime-only content would also now be more purposefully integrated alongside other related reporting.

Additional objectives included a better mobile experience, faster and more frequent Livewire posts, improved site performance, more intuitive navigational tools, and an overall cleaner reading experience.

It’s likely that regular readers visiting the new site for the first time might not have found all these objectives fully realized. This isn’t just a matter of redesigns being notoriously agonistic to habitual readers. It’s also the result of the editorial learning curve inherent in the publishing tool we built. Despite weeks of beta testing, its next to impossible to simulate the real-life environment of a newsroom, especially when news is breaking. We’ve added a lot of power in the new system, but it will take some time to figure out how best to wield it.

The new site isn’t simply a redesign, its a fundamental change in the way we publish and present stories. Unlike a more traditional redesign where we might be locked into a certain layout, NEO allows us to create a dynamic and flexible page where the layout changes to reflect our coverage of the day’s news. NEO also makes iterating and adding new features a relatively simple process. We’re already working on version 1.1, including several reader-suggested features, and in the future readers can expect much more frequent updates to our site’s design than we’ve had in the past.

There’s one final point worth making on the role that technology and design play in a newsroom. While it’s understandable that most readers will have only passing interest in how their news gets made, in an increasingly digital news environment the decisions publishers make regarding their publishing technology can have critical consequences. Over the past few years we’ve witnessed a number of publishers dependent on large tech platforms like Facebook and Medium for their publishing infrastructure be forced to close their doors. Similarly, the homogenization in news presentation has contributed to the rise of “fake news.” But, it’s not all doom and gloom.  

Publishers who have been able to integrate tech or product teams into the editorial process and develop tools specific to their audience have seen notable benefits, particularly in areas like membership growth.

While it can be tempting to view tech and editorial as two distinct parts of digital publication, at the most successful organizations tech and editorial work in tandem and share a unified vision of the product. Among the six primary members of the TPM product team, we have a combined 25 years experience working at TPM. A project like NEO would simply not be possible without this deep understanding of TPM’s editorial processes and objectives.

NEO was built entirely in-house and is the result of the incredible work of our Director of Technology and lead developer Matt Wozniak, UI/UX developer Jacob Harris, and designer Christine Frapech. Thanks are also due to our publishing associate Jackie Wilhelm, Executive Publisher Joe Ragazzo, Publisher Josh Marshall, Managing Editor David Kurtz, and our entire editorial staff.  And most of all, thanks to our readers, especially the nearly 30,000 Prime members — without you ambitious journalism projects like this couldn’t happen.