While many of us marvel at the beautiful and creative data designs and Google mashups that pepper sites like FlowingData, 10000words, Smashing Magazine and The Next Web, those of us seeking to reproduce such designs for our agencies are plagued by plenty of challenges. There's mastering the software needed to do these things, collecting the right data in the right format, and finding a way to create data visualizations fast enough to keep up with the breakneck pace at which the federal government is putting out data. 

Agencies have always been inundated with information and struggled with ways to extract data to better inform decision makers. Today, they are not only challenged with their internal informational needs, but they are also saddled with security procedures and expectations of the new Administration's goals to make government data available to the American public through its transparency agenda.

Enter the mashup.

The key idea behind "mashups" is the ability to easily integrate information from a wide variety of sources and make it useful for a wide variety of users. It's strength lay in taking two or more types of information — for example, maps and store locations — and combining them or mashing them together into one.

In the old days, GIS (geographic information system) was the only tool that allowed you to accomplish such a feat. Today, the software landscape is quite different. In fact, if you have ever used programs like Google Earth or Microsoft's Virtual Earth, you have probably used a mashup — maybe without even knowing it.

Nimble and quick

One company, JackBe, thinks mashups — especially given the push to get out government data on data.gov — is the wave of of the future, and is betting big on their own mashup making platform.

OhMyGov! spoke to John Crupi, the Chief Technology Officer for JackBe, to discuss how his company's work in mashup software can help improve government decision-making.

"There were two big problems we wanted to solve," Crupi said, "how to get the data and how to share the data in context."   

A 20-year veteran in object oriented and enterprise distributed computing, Crupi lived through the changes in the IT industry before broadband and wireless arrived, when applications were run on local PCs. He reflected on the late 1990s as a time of "rich applications that could not be run on the browser."

"Ten years ago we could not do what we are doing with JackBe," he said. "However, with web standards and governance we don't have to write custom adapters anymore. Now, JackBe users can create many types of mashups which can be shared with co-workers through spreadsheets, portals, and mobile devices."

The center of JackBe's product offering is an enterprise mashup server (Presto Edge) that supports the user-driven ad-hoc integration of data.

In the lingo Presto uses, information (filtered and selected data) in a format ready for a user interface is the 'mashup.' The display of mashup data is another element, typically a Presto "mashlet"— a mini application or widget that uses mashup data. Mashlets package both data and user interface, and can be embedded and used as-is in Web pages, wikis and mobile applications.

How can this technology help government agencies?

"Look at the TARP stuff," Crupi says. "You may look at one slice and say, 'here's what we allocated and here's their payback schedule.' But what they want to do is say, 'Wait a second, what are the other things these banks are doing.' We want to see if there's insider trading; we want to see if their are SEC investigations; we want to see if people are on a different board. When you put the information together, it gets you this 360-degree view. The government calls this situational awareness." 

One of JackBe's first government customers was the Defense Intelligence Agency, but they did not have an easy entry into the public sector. For example, Crupi described government's "No trust and no need to share" mentality as a historical obstacle. In the past, there was no incentive to trust and share and it was always difficult to come into an organization where their livelihood is based on data, according to Crupi.

But smarter and less paranoid minds prevailed, of all places, in DoD's intelligence community. In 2008, JackBe was awarded a contract to create a situational awareness solution for the Defense Information Systems Agency (DISA). The tool was a virtual operating center that aggregates data from hundreds of current and future disparate services on classified and unclassified networks into a customizable, desktop-like intelligent asset dashboard for the real-time collection and analysis of data.

Currently, JackBe is plugged into the government like many within industry as more and more request for proposals (RFPs) leave agency offices to be competed for agile dashboards like that used by the recently unveiled Recovery.gov.

The key to gaining the government's trust, says Crupi, "is not to be intrusive with their current architecture... you need to be pluggable and interoperable." He added, "Then you need to work within their security and be able to address identity management."

Own, but share alike 

The seemingly in-bred resistance to sharing data is not surprising to those working with federal agencies. And it's been a challenge for the data.gov project.

"Agencies want to brand their data," Crupi says. "Their data is the only way to show performance," since by their nature most government agencies do not generate revenue. Without interesting data to claim as their own, they get worried that Congress will cut their funding for research. "You go into DOD and they have marketing teams; they have pens, they have mugs. They have to hype it up. It's about money."

One problem with government data is that agencies don't know what's being used and by whom. "If you lop the head off of something and there is no longer a way to identify the data, all of a sudden you have a big identity problem," he says. So JackBe built identity management into its product. "Because we're sitting in the middle, we can track which data is being used more often and by who ... so that teams can show usage."

As for access and security, especially in sensitive environments, Crupi says that JackBe "allows for creators to put policies in place that says if we put two things together, it changes the authorization level. As you are maneuvering through the system it knows who you are and is checking to see if you have access to this data."

Because mashups create new data sets, there is an inherent risk in accidentally creating security risks. For example, a list of all the U.S. military bases around the world by name is not considered highly secretive. But place those names on a Google map mashup, and all of a sudden you have created a target list for U.S. enemies of state. Hence the need for identity management.

Founded in 2002 as an AJAX widget company, JackBe is quick to disclose they are not a service company, but a product company.

"A lot of our customers build the mashups," said Crupi. "We're not a service org. We're a product company."

Emailing spreadsheets is not real-time collaboration  

And the company is not without competition. There are other heavy hitters like IBM, Microsoft, Google and Yahoo in the mashup space. But for the meantime, JackBe seems to have the edge in the government mashup business, as they continue to partner with larger firms and provide their niche product. 

"Decision makers are struggling to get the info they need to make decisions, so they are making decisions based upon partial information," Crupi says. "We can't be emailing Excel spreadsheets as a means of collaborating."

Speed remains an issue too. "Being able to be very iterative and incremental isn't something the government has their arms around yet. They are use to long projects — things taking years and decades to do.

"The ultimate goal," Crupi says, "is to get at trapped data quickly so that you can make informed decisions. You should know in hours or days that what you're doing is not correct. It's too costly to find out what you did 6 months or a year down the road was wrong."

Read about more cool technologies for government in our Innovations section