Stephen E. Arnold: Search Big Data Flim-Flam – And One Open Source Search of Compressed Files with SQL (RainStor)

Categories: Data
Stephen E. Arnold

Stephen E. Arnold

Search and Big Data: Been There, Done That

Is the use of search to find information in large collections of content revolutionary? Er, no. What about using search to locate an Internet Protocol address in a repository of monitored email traffic? Er, no.

With the chatter on LinkedIn and the vacuous news releases from some floundering search companies, one would think that gathering up content and running a query was the equivalent of my ancestor stealing and ember and saying, “Look, I invented fire.”

Sorry.

Beyond the rather influential if specious IBM white paper published in 2010 (link is at http://bit.ly/1gckiPJ), a large number of companies continue to position some old as new again.

One interesting twist on the “search is better than SQL” is the useful solution brief from RainStor. In some circles, RainStor has a low profile. In others, the company has caught the attention of some recognized “names” in the Big Data world; for example, Cloudera and Dell. So think Hadoop friendly.

Read the rest of this entry »

Comments Off
Apr 13

Yoda: US Government Gets It Right on Open Data

Categories: Data
Got Crowd? BE the Force!

Got Crowd? BE the Force!

Open Data Law Aims To Demystify Federal Spending

Landmark DATA legislation, near passage, would transform the US government’s spending information from a maze of confusing documents into easily accessible open data.

Both houses of Congress stand poised to pass a law three years in the making that would standardize how the federal government’s spending data are published. The Digital Accountability and Transparency Act (DATA Act), regarded by some observers as the most significant open-government legislation since the Freedom of Information Act in 1966, would transform the US government’s spending information from a maze of confusing documents into easily accessible open data.

The new law would require the federal government to automate, standardize, and publish its myriad financial management, procurement, and related data in electronic formats that can be easily accessed and analyzed by interested parties in the public and private sectors.

Read full article.

Comments Off
Apr 7

Stephen E. Arnold: New York Public Library Posts Open Maps

Categories: Access,Data,Geospatial
Stephen E. Arnold

Stephen E. Arnold

New York Public Library Posts Maps

The New York Public Library has a massive collection of beautiful maps, but instead of keeping them locked in an archive Motherboard reports, “The New York Public Library Releases 20,000 Beautiful High Resolution Maps.”

All of the 20,000 maps are available via open access. What is even more amazing is that the NYPL decided to release the maps under the Creative Commons CCO 1.0 Universal Public Domain Dedication. If you are unfamiliar with a Creative Commons license, it means that users are free to download content and do whatever they want with it.

Read the rest of this entry »

Comments Off
Apr 6

Stephen E. Arnold: USG Considers Open Acquisition

Categories: Data,Governance
Stephen E. Arnold

Stephen E. Arnold

Government Tackles Acquisition Inefficiencies

Given evidence like the vile backlog on veterans’ benefits and the still-operating paperwork bunker in Pennsylvania, one could be forgiven for suspecting that no one in government is even trying to bring our bureaucracy into this century. You may be surprised to know there is plan in place for at least part of the problem, as evidenced by the Integrated Award Environment: the Path Forward from the U.S. General Services Administration (GSA). That document, which looks suspiciously like a Power Point presentation converted to PDF, outlines the GSA’s recommendations for improving the federal government’s acquisition procedures.

Anyone interested in the details should check out the document, but the list of “our principles” summarizes the organization’s targets:

Read the rest of this entry »

Comments Off
Apr 6

Stephen E. Arnold: Open Data Collection with Ushahidi

Stephen E. Arnold

Stephen E. Arnold

Open Data Collection with Ushahidi

March 28, 2014

The crowdsourced data collection platform Ushahidi, now assisting activists worldwide, was first created to facilitate public accountability and social activism during crises in its home nation, Kenya. Not surprisingly, Ushahidi is also the name of the non-profit behind the open-source project. Open-Steps.org interviewed the organization’s director of data projects, Chris Albon, about the platform. The article prefaces the dialogue with a brief explanation:

“In a nutshell, it allows citizens to make reports in a collaborative way, creating crowdsourced interactive maps. With a very intelligent approach, Ushahidi gives citizens the possibility to use the web, their smartphones and even SMS to gather data, which makes this technology accessible almost everywhere and for everyone. Originally created in Kenya to serve as an instrument for social activism and public accountability in crisis situations, the software has proven to be a great companion worldwide in bringing advocacy campaigns to a successful end. The team behind Ushahidi has not only created a world-changing technology but also they share it with others since it is released as Open Source.”

Albon tells us that the core Ushahidi platform is now being used in 159 countries and has been translated into 35 languages, and explains it is being used by groups from small, election-monitoring non-profits to global organizations tracking disaster relief efforts. Journalists also make use of the platform. Albon notes that his group helped build iHub in Nairobi, an “innovation hub” and community workspace designed to facilitate collaboration and community growth. See the article for more on this and Ushahidi’s other projects, Crowdmap, Swiftriver, Ping, and BRCK. The interview wraps up with something to look forward to: the next generation of the Ushahidi core platform, v3, is on its way.

Cynthia Murrell, March 28, 2014

Sponsored by ArnoldIT.com, developer of Augmentext

Comments Off
Mar 28

Stephen E. Arnold: OpenText Innovates In File Sync And Share

Stephen E. Arnold

Stephen E. Arnold

OpenText Innovates In File Sync And Share

OpenText announces its new product: OpenText Tempo. The new file sharing collaboration platform is highlighted in the press release, “Social Collaboration Combined With Secure Files Sync and Share: Introducing OpenText Tempo.” OpenText describes Tempo as a project that required teamwork from all over the world.

OpenText Tempo will be able to:

“…provides an engaging user experience that combines the convenience of secure file sync and share with social collaboration and seamless integration to Content Server. It connects people with each other and with their content in a secure, compliant environment that enables open dialogues to take place, extending the value of content through the process of collaboration.”

It is the company’s first EIM application with integration for other products, including Tempo Note, Tempo Social, and Tempo Box. OpenText also says it improves Web site management, web experience management, and portal applications.

It is a commercially secure file sharing and social platform. Will Dropbox and other free services be able to something similar on at an appealing price point?

Whitney Grace, March 01, 2014
Sponsored by ArnoldIT.com, developer of Augmentext

Comments Off
Mar 1

Patrick Meier: Crisis Mapping without GPS Coordinates

Patrick Meier

Patrick Meier

Crisis Mapping without GPS Coordinates

I recently spoke with a UK start-up that is doing away with GPS coordinates even though their company focuses on geographic information and maps. The start-up, What3Words, has divided the globe into 57 trillion squares and given each of these 3-by-3 meter areas a unique three-word code. Goodbye long postal addresses and cryptic GPS coordinates. Hello planet.inches.most. The start-up also offers a service called OneWord, which allows you to customize a one-word name for any square. In addition, the company has expanded to other languages such as Spanish, Swedish and Russian. They’re now working on including Arabic, Chinese, Japanese and others by mid-January 2014. Meanwhile, their API lets anyone build new applications that tap their global map of 57 trillion squares.

When I spoke with CEO Chris Sheldrick, he noted that their very first users were emergency response organizations. One group in Australia, for example, is using What3Words as part of their SMS emergency service. “This will let people identify their homes with just three words, ensuring that emergency vehicles can find them as quickly as possible.” Such an approach provides greater accuracy, which is vital in rural areas. “Our ambulances have a terrible time with street addresses, particularly in The Bush.” Moreover, many places in the world have no addresses at all. So What3Words may also be useful for certain ICT4D projects in addition to crisis mapping. The real key to this service is simplicity, i.e., communicating three words over the phone, via SMS/Twitter or email is far easier (and less error prone) than dictating a postal address or a complicated set of GPS coordinates.

Source with Graphics

Comments Off
Feb 25

Berto Jongman: How visualising data has changed life…and saved lives

Categories: Data,Design
Berto Jongman

Berto Jongman

How visualising data has changed life… and saved lives

From John Graunt’s ‘bills of mortality’ to Florence Nightingale’s revolutionary ‘rose charts’, the distillation of information into graphics has been a vital tool for scientists

Nicola Davis

The Observer, Saturday 15 February 2014

Big data, infographics, visualisations – the pop words of a modern phenomenon. But while information accumulation has become a 21st-century obsession, our generation is not the first to discover that a picture is worth a thousand words, as a new British Library exhibition will reveal.

Revelling in the power of illustrations, tables and figures, Beautiful Science charts the course of data dissemination across the centuries, from the grim ledgers of death recorded by John Graunt in the 17th-century “bills of mortality” to the digital evolutionary tree dreamt up by an Imperial College researcher, complete with a mind-boggling zoomable function. “You can use almost fractal-like patterns to explore all of life on Earth,” says Dr Johanna Kieniewicz, lead exhibition curator.

Read the rest of this entry »

Comments Off
Feb 16

Neal Rauhauser: Global Database of Events, Language & Tone (GDELT) Is SAFE!

Categories: Data
Neal Rauhauser

Neal Rauhauser

Global Database of Events, Language & Tone (GDELT) Is SAFE!
by Neal Rauhauser

That was a long, uncomfortable silence, after I posted GDELT’s Mysterious Demise, but we now have the particulars on what happened:

The bottom line is that GDELT is one of the very few event datasets in existence today that actually has all of the necessary permissions.

Read the rest of this entry »

Comments Off
Jan 26

Worth a Look: Beyond Transparency – Open Data and the Future of Civic Innovation

Comments Off
Jan 23