Splunk dabbles in edgy hardware, lowers data ingestion

Splunk has released a major update to its core data-crunching platform, emphasizing reductions in the quantity of data ingested and therefore the cost of operations.

It also addresses a few security flaws that may not be fixable in earlier editions. The release is called Splunk 9.0.

As explained to The Register by Splunk senior vice president Garth Fort, the changes reflect users' concerns that Splunk sucked up so much data that using the application had become very expensive. Fort even cited a joke that did the rounds when Cisco was said to have $20 billion earmarked to spend on Splunk and observers couldn't be sure if that was the sum needed to buy the company or just pay for licences.

Version 9.0 is designed to address that issue by allowing users to sort, filter, redact or otherwise manipulate data before it’s ingested into Splunk. Doing so reduces the amount of data going in, so reduces the cost of operations.

Fort said one reason for this feature is the increasing use of edge computing, an environment in which plenty of data is created but not all of it will be worthy of transport to a cloud.

Splunk has therefore created hardware – known in-house as "the puck" – that does some pre-processing of data on the edge. Fort said Splunk has demoed the device to customers including Royal Dutch Shell and received very positive feedback. For now, the company has no plans to productize "the puck" and Fort said Splunk would likely partner with an established hardware provider rather than build the product itself.

Another feature added to Splunk 9.0 allows use of external cloud storage. Splunk has already allowed users to move data into different instances. Now the sort of data that is considered unworthy of ingestion into Splunk can be sent to cloud storage but still be searched from within the company's software.

Fort said this "federated search" feature means users are spared the need to acquire another tool to give them a single view of data in Splunk, and beyond.

• Splunk CEO jumps ship, share price slumps despite surging growth
• Splunk spots malware targeting Windows Server on AWS to mine Monero
• Splunk junks 'hanging' processes, suggests you don't 'hit' a key: More peaceful words now preferred in docs

Version 9.0 has been significantly re-architected to address security issues that Fort said will be detailed after its launch. He mentioned a handful of significant flaws will be revealed, and that version 9.0 fixes them but not all can or will be patched for users of previous versions of the company’s flagship software.

Fort said Splunk will conduct more than usual activity encouraging users to upgrade, including adding error messages to older versions of the software. Past major Splunk releases have seen 30 percent of users upgrade within six months, he said. The company hopes the move to version 9.0 will be faster. Users of cloudy instances are exempt – their rigs will be updated automagically.

Cloud users will also get to play with "Splunk Assist", a new service that inspects Splunk instances and suggests improvements. Fort mentioned insights such as soon-to-expire certificates.

Another new product, in preview, is called "Splunk Cloud Developer Edition" and apparently speeds the process of developing apps for the Splunk Cloud Platform. And "Anomaly Detection Assistant" uses ML to help "security analysts, IT operations, and DevOps engineers find potential problems by using machine learning to craft a perfectly tuned query quickly in order to identify anomalies in a time-series datasets."

Splunk Enterprise Security gains risk-based alerting that the vendor says makes it easier to "enforce a zero trust approach, prioritize high-fidelity incidents, and ensure rapid time to action by automating containment and response tasks." ®