Public Intelligence Blog

Stephen E. Arnold

Stephen E. Arnold: The New Landscape of Enterprise Search. A Critical Review of the Market and Search Systems. Published by Pandia, Oslo, Norway, 2011.
ISBN: 978-82-998676-0-3

Pandia Book Page

The following enterprise search companies are covered in detail:

Autonomy
Endeca
Exalead
Google Search Solutions
Microsoft and Fast
Vivisimo

Book Page

Phi Beta Iota:  For over fifteen years Steve Arnold has been the “virtual CTO” for OSS.Net (ceased operations at end of 2010) and Earth Intelligence Network, of which he is a founding partner.  We have reviewed the 154-page document, and as with all publications by Steve Arnold, author of the still-trenchant Google Trilogy, have found it to be deeply helpful including sharp but polite eviseration of both Google and Microsoft (Microsoft comes out ahead of Google, but has its own sucking chest wounds).  What is clear to us is that none of the major vendors are serious about  the necessary migration to open source software that integrates information sharing, multi-media analytics, cross-disciplinary directories, and deep web applications, for example, integrating c drives and emails across all poverty stakeholders.  The big new trend in this book–and in our own research–is the compelling nature of five minute videos as documents of lasting value.  We are seeing this in open education and non-government decision-support.  Combine that with on demand translation and the way is open for someone to create the M4IS2 system of the future.  We doubt it will be Google, and we are certain it will include OpenBTS for the three billion poor, the center of gravity for future knowledge.

The search term brings up appropriate results, but the fact of the search gives us an opportunity to provide comment.

1)  Nothing now being used by governments, and certainly not iBase or Palantir, both aging technologies that do not scale and have too many fat-finger handicaps, fulfills the originial requirements documents crafted in the late 1980′s.

Worth a Look: 1989 All-Source Fusion Analytic Workstation–The Four Requirements Documents

2)  The ONLY programs that have gotten anywhere close are COPERNICUS plus plus, and SILOBREAKER.  However, both of these have been slow to recognize the urgency of integrating–fully integrating–capabilities that address each of the eighteen functionalities.  Below is the list of softwares now in use by US Special Operations Command J-23 Open Source Intelligence Branch along with the STRONG ANGEL TOOZL and a couple of other things.

Memorandum: USSOCOM Software List and STRONG ANGEL TOOZL

See also:

Definitions: M4IS2 (Multinational, Multiagency, Multidisciplinary, Multidomain Information-Sharing & Sense-Making

Search: The Future of OSINT [is M4IS2-Multinational]

Worth a Look: Deep Web Multilingual Federated Search

1988-2009 OSINT-M4IS2 TECHINT Chronology

Worth a Look: Planetary Skin Data Sharing Initiative

Search: meta-tagging humint

Who’s Who in Librarian Intelligence: Arno Reuser

Who’s Who in Public Intelligence: Mats Bjore

Who’s Who in Collective Intelligence: Steve Arnold

Journal: Dr. Dr. Dave Warner Shares…

Event Report CORRECTED LINKS: Responding to Real Time Information, Open Systems and the Obama IT Vision [Google-Microsoft Meld]

Review: The Starfish and the Spider–The Unstoppable Power of Leaderless Organizations

Review: Innovation Happens Elsewhere–Open Source as Business Strategy

Journal: Google, the Cloud, Microsoft, & World Brain

Worth a Look: GeoChat (SMS Plotted on Map)

2006 Yekelo (ZA) Continental Early Warning & Information Sharing: A Military Perspective on Deterring & Resolving Complex Emergencies

1998 Arnold (US) New Trends in Automated Intelligence Gathering Software

The global standard for multinational information-sharing and sense-making is in the process of being designed, funded, and distributed.  If you think you have something relevant to that, generally only open source software will be considered, get in touch with any of the individuals above.

Directory & Links

Description: A hot linked and annotated list of social search systems.

Feature text below:

One of the first steps in Strategic Social Networking is to know where to look for information from social systems. Google, Microsoft, and Yahoo include content from high-profile social content sources like Facebook. Here at SSN, we need to look for fresh social information from different angles, from different services.

Phi Beta Iota: Steve Arnold has been the “virtual CTO” for the modern Open Source Intelligence (OSINT) community and is generally ten years ahead of the private sector and twenty years ahead of any government.

Paradise Found

The future of OSINT is M4IS2.

The future of Open Source Intelligence (OSINT) is Multinational, Multifunctional, Multidisciplinary, Multidomain Information-Sharing & Sense-Making (M4IS2).

The following, subject to the approval of Executive and Congressional leadership, are suggested hueristics (rules of thumb):

Rule 1: All Open Source Information (OSIF) goes directly to the high side (multinational top secret) the instant it is received at any level by any civilian or military element responsive to global OSINT grid.  This includes all of the contextual agency and mission specific information from the civilian elements previously stove-piped or disgarded, not only within the US, but ultimately within all 90+ participating nations.

Rule 2: In return for Rule 1, the US IC agrees that the Department of State (and within DoD, Civil Affairs) is the proponent outside the wire, and the sharing of all OSIF originating outside the US IC is at the discretion of State/Civil Affairs without secret world caveat or constraint.  OSIF collected by US IC elements is NOT included in this warrant.

Read the rest of this entry »

Steele as Wild Card Speaker

Phi Beta Iota: This started as a short list for the various college and university engineering students that have dialed in, but as we got into it, it became more of a “situational awareness” inventory pulling together both technical “solutions” none of which have been integrated yet, and analytic “requirements” none of which have been satisfied yet.

$75 billion a year for secret intelligence, and we still do not have an analytic desktop toolkit, all-source geospatially and historically and cultural astute back office processing, or global reach to all humans, all minds, all the time.  Sucks for us.  Let’s see what the Smart Mob can do….

We specifically invite suggestions in the Comments for removing items from this list, or for adding items from within this website or from any other website.  This is now a work zone.  Steele is available to visit any engineering workshop and especially those working on bottom-up clouds like Swarm DPL (transparently scalable distributed programming language).

See also About the Idea, Articles, Briefs, Handbooks, and Historic Contributions. Use the Menu–everything on this stie responds to “OSINT” that is not a good search term.  Rock on!

TOTALLY TECH:

2009 Arnold Google: The Digital Gutenberg

2008 Memoranda: Policy-Budget Outreach Tool

2006  USSOCOM Software List and STRONG ANGEL TOOZL

2006 Morville (UK) Ambient Findability Massive Scale Beyond Your Imagination A Recommedned Approach to Creating the World Brain with Instant Recall

Read the rest of this entry »

FIXED Big File Problem.  Three Separate Files, each at Their Respectuve Thumbnails.

Somat Engineering (8A) CHANGE 2010: Responding to Real Time Information, Open Systems and the Obama IT Vision

EVENT REPORT: Have you been wondered just how we are going to implement M4IS2* and add to it Real-Time-Processing (RTP) and Near-Real-Time Processing (NRTP)?  This event introduced to the Washington, D.C. environment, for the first time, the concept of operations and related technologies that will allow Google and Microsoft to play well together within any existing desktop analytic environment that relies on Microsoft for the operating system and functionalities, but needs to or wants to rely on Google for everything else.  This was the first of a quarterly series of briefings organized and moderated by Steve Arnold, the behind-the-scenes architect of FirstGov (now Gov.USA), and author of the three definitive analytic studies of Google as well as multiple studies of Microsoft.

OSS.Net, Real-Time Information (RTI): Injecting Now Information into Decision Systems

* Multinational, Multiagency, Multidisciplinary, Multidomain Information-Sharing & Sense-Making.  Precursor to the United Nations Open-Source Decision-Support Information Network (UNODIN).

Click on Gorilla to see three briefs in one document with notes, as delivered less several proprietary slides, on 23 September 2009 at the National Press Club. Ram Ramanujam, President of Somat Engineering, and Mark Crawford, Project Manager for Somat Engineering, were also present.  The Washington office of this 8A ICT firm is managed by Arpan Patel.

Adhere Solutions, Leveraging Google for Information Integration and Openness

All briefings can be read in Notes format for selected planned words intended to accompany most slides.

Below are biographies of the four principals.

Read the rest of this entry »

Real-Time Information

Under the direction of maestro Steve Arnold, and in support of two extraordinary companies, Somat Engineering represented by its CEO, G “Ram” Ramanujam and it Washington DC Director Arpan Patel; and Adhere Solutions represented by its co-founder Jim Orris, a 15-slide 8-minute presentation was crafted to address Real Time Information and its injection into decision systems.

The Good Part I

Watch the Video (1.3 Hours)

Good on Background, Hop in and Out for Screen Shots)

Robert Steele’s Notes:

Read the rest of this entry »

Hacker Steele

Changes to the Earth that used to take 10,000 years now take three years.  This has led to a very strong interest in “real-time science.”  At the same time, as the information society matures, we are seeing demands for government information (paid for by the taxpayer) to be made available in real-time.  Census results that used to take ten years to process now take two years, and that is major progress.  It is also 20 years behind the art of the possible and some would say, the art of the necessary.   With Rapid SMS where it is today in Africa, there is no reason why census data–and all other forms of data funded by the taxpayer–cannot be made available as collected, as processed, as analyzed–three different levels of value, none now constrained by time and materials, only by old mind-sets unfamiliar with the state of best practices outside the wire.

Similarly, real-time marketing and real-time needs definition and satisfaction are emergent.

Steve Arnold, for over a decade the “virtual CTO” to the multinational multifunctional information-sharing and sense-making community that gathered annually from 1992 to 2006, is at the forefront of this specific emerging convergence of human needs and information communication technologies (ICT). Below are snippets from three of his recent pieces, and contact information.

Read the rest of this entry »

Google Evil--Bad Google

To begin on a positive note: Google has both computational mathematics that are out of this world, an order of magnitude better than anything IBM or the other usual suspects can muster, and it has cracked the cloud storage and heat and throughput issues in a way that is now an almost insurmountable barrier to entry for corporations (China and India are emergent and Brazil and Russia might have some anti-Google surprises in the making).

Google lacks human and intelligence-oriented leadership, and suffers from the usual problems associated with a bureaucracy that grew too fast on the fantasy cash from credulous investors.  “Zooglers” is the term of art now used for Google people that have vested and leave to create new capabilities that “surf” on Google while doing things the Google bureaucracy could not countenance as an internal active.

Then there is evil Google.  Click on the Google Evil-Bad Google logo for a page of links, or choose from among the links below.  We know Larry Page personally, through the Silicon Valley Hackers Conference.  He is a nice person, an engineer, with zero ethics in the sense that dorks do not know anything about etiquette.  Eric Schmidt is the new monopolist in town, and Eric Schmidt has absolutely zero interest in the public interest.

Read the rest of this entry »