Speculus Threat Intelligence for Splunk

Prerequisites

You need an active Speculus subscription. Your subscription provides an MMDB endpoint URL and an API key, both of which you enter during app setup. Reach out to [email protected] if you need one.

The app downloads a copy of the Speculus combined threat database and enriches IPs locally against it. It does not call an API at search time.

Privileges

• The user installing the app needs admin_all_objects.
• The user configuring the updater input needs permission to create data inputs and write stored credentials, which the admin role has by default. On Splunk Cloud this is sc_admin.
• Search users need no special privileges. The speculus command reads only the local database file and never touches stored credentials, so anyone who can run a search can use it.

Where to install

Install on your search head only. Indexers need neither the app nor the database. The search command is non-distributed, so the database is never shipped in the knowledge bundle and you do not have to size or synchronise a large binary file across your indexing tier.

In a search head cluster, deploy through the deployer rather than installing on individual members.

Allow about 200 MB of free disk space on the search head for the database and its updates.

Installation

Install from Splunkbase

1. Download the application file from Splunkbase: splunkbase.splunk.com/app/9081
2. From Splunk click on Apps > Manage Apps.
3. Click Install app from file.
4. Upload the compressed file.
5. Restart Splunk when prompted.
6. Complete the app setup. This requires your Speculus endpoint URL and API key.

Install from the Splunk app store

1. From Splunk click on Apps > Find more apps online.
2. Search for Speculus.
3. Click Install.
4. Restart Splunk when prompted.
5. Complete the app setup. This requires your Speculus endpoint URL and API key.

Manual installation

1. Obtain the application file from Speculus.
2. From Splunk click on Apps > Manage Apps.
3. Click Install app from file.
4. Upload the compressed file.
5. Restart Splunk when prompted.
6. Complete the app setup. This requires your Speculus endpoint URL and API key.

Or from the command line:

$SPLUNK_HOME/bin/splunk install app /path/to/speculus_ti-0.3.0-release.tar.gz -auth admin:<password>
$SPLUNK_HOME/bin/splunk restart

A restart is required. The speculus search command and the updater input are not registered until splunkd restarts.

App setup

The app installs without a threat database. Point it at your Speculus endpoint once and it keeps itself current from then on.

1. Open the Speculus Threat Intelligence app.
2. Click Speculus Setup in the navigation bar, which links straight to the input. You can also reach it at Settings > Data Inputs > Speculus MMDB Updater > New.
3. Give the input a name, for example default.
4. Enter your Endpoint URL, for example https://mmdb.example.com/v1/mmdb.
5. Enter your API key.
6. Leave Verify TLS certificate enabled. Disable it only when testing against a self-signed endpoint.
7. Optionally check More Settings to set the interval, default 14400 seconds (4 hours), or the index for status events.
8. Click Next.
9. Enable the input. It ships disabled so it never runs before it is configured.

The first download begins within one interval, or immediately if you restart Splunk. The database is roughly 90 MB, so allow a few minutes for the first run.

About your API key. The key is entered once, then moved into Splunk’s encrypted credential store on the first run and masked in inputs.conf. It is not left on disk in plain text. Because it is read from secure storage afterwards, a key rotated upstream must be re-entered on the input rather than edited in a file.

Verify the installation

| makeresults | eval ip="1.1.1.1" | speculus field=ip

A row with speculus_matched and a set of speculus_* fields means the app is working.

Check the updater is healthy:

sourcetype="speculus:mmdb_updater" earliest=-24h

Every run writes a status event saying whether the database was updated or already current. Failures are written as events too, with the reason, so a bad key or unreachable endpoint is visible without reading logs.

How updates work

Once configured, the app maintains itself.

Each run sends a conditional request using the database’s ETag. If nothing has changed, the run costs a single round-trip instead of re-downloading 90 MB.

A newly downloaded file is opened and validated as a real database before it replaces the live copy. A truncated or corrupt download is discarded, so a failed update cannot break searches that already work.

The active database lives at:

$SPLUNK_HOME/etc/apps/speculus_ti/local/data/speculus_combined.mmdb

It sits under local/ because that is runtime state. App upgrades will not overwrite it, so you do not re-download the database on every upgrade.

Upgrading

1. Install the new package as above, ticking Upgrade app on the upload form or passing -update 1 on the CLI.
2. Restart Splunk.

Your input configuration, stored API key, and downloaded database all live under local/ and survive the upgrade. Nothing needs reconfiguring.

Search commands

Append speculus to any search that has an IP field. It adds enrichment fields to every event and passes the events through, so you can filter, aggregate, or chart on the results.

Arguments

Both are optional.

• field names the field holding the IP address.
• prefix changes the prefix on the added fields. Defaults to speculus_. Use it when you enrich twice in one search, or when speculus_* would collide with fields you already have.

If you leave out field, the command finds the IP itself, using the first of these present on the event:

src_ip, dest_ip, src, dest, clientip, client_ip, remote_addr, remote_ip, ip

So most searches need no arguments at all:

index=firewall | speculus

IPv4 and IPv6 both work. Every event gets the full field set whether or not the address was found, so table columns and stats stay stable. An unknown address returns speculus_matched="false" with empty values rather than being dropped.

Common searches

Look up a single address

| makeresults | eval ip="1.1.1.1" | speculus field=ip

Filter a search down to risky sources

index=firewall | speculus field=src_ip | where speculus_score >= 75

See the risk profile of your traffic

index=web | speculus field=clientip | stats count by speculus_risk

Find anonymizing infrastructure

index=firewall | speculus field=src_ip
| where speculus_is_tor="true" OR speculus_is_vpn="true" OR speculus_is_proxy="true"
| stats count by src_ip, speculus_org

Find known threat infrastructure in your logs

index=firewall | speculus field=src_ip
| where speculus_threat_activity!=""
| stats count by speculus_threat_activity, speculus_threat_malware
| sort - count

Hunt residential proxy abuse

Residential proxies are how credential stuffing and scraping hide behind ordinary-looking ISP addresses.

index=web | speculus field=clientip
| where speculus_proxy_type="residential"
| stats dc(clientip) as ips, count as requests by speculus_proxy_provider
| sort - requests

Rank the riskiest sources hitting you

index=firewall | speculus field=src_ip
| where speculus_matched="true"
| stats count as hits, values(speculus_threat_activity) as activity by src_ip, speculus_country, speculus_org, speculus_score
| sort - speculus_score, - hits
| head 50

Enrich efficiently over large volumes

Deduplicate before enriching when you care about the addresses rather than every event.

index=firewall src_ip=*
| stats count as events by src_ip
| speculus field=src_ip
| where speculus_score >= 50
| sort - speculus_score

Enrich both ends of a connection

index=firewall | speculus field=src_ip prefix="src_ti_" | speculus field=dest_ip prefix="dst_ti_"

Alert on high-risk traffic

Save any enriched search as an alert.

index=firewall | speculus field=src_ip
| where speculus_score >= 90 OR speculus_threat_activity="C2"
| table _time, src_ip, dest_ip, speculus_score, speculus_threat_activity, speculus_org

Working with the values

Booleans come back as the strings "true" and "false":

| where speculus_is_tor="true"

Numbers come back as strings. where comparisons handle that, but convert explicitly for arithmetic or charting:

| eval speculus_score=tonumber(speculus_score) | stats avg(speculus_score) by index

An address can carry proxy detail, threat detail, both, or neither. Fields for an absent category come back empty, so test with !="" rather than isnotnull():

| where speculus_threat_activity!=""

Dashboards

Speculus Threat Feed

Point it at an index and it tells you which addresses in that data are worth attention.

Choose the index, the IP field to scan, and a minimum score. The View selector narrows to threat IOCs only, anonymizers, or compromised hosts.

You get counts of high-risk IPs, threat IOCs, Tor nodes and anonymizing VPNs or proxies, a map of where connections originate, breakdowns by threat activity, risk level and source organization, and a table of observed risky addresses. Click any address in the table to pivot into investigation.

Start here when the question is “is there anything bad in this data?”

Speculus IP Investigate

Everything known about one address. Threat score and risk level, what the address has been observed doing, its network and geographic origin, infrastructure flags, and the full threat and proxy records with first and last seen dates.

It also shows which of your own indexes contain that address, and its sightings over time. That is the part an external intel feed cannot tell you: not just that an address is malicious, but whether it has been touching your environment, and since when.

Start here when the question is “what is this address, and have I seen it before?”

Right-click on any IP

Two actions appear on IP fields throughout Splunk, in both the event viewer and the field menu. No search syntax needed.

• Speculus: Scan IP opens IP Investigate for that address.
• Speculus: enrich this IP in search runs its full enrichment record in a new search.

They attach to ip, src_ip, dest_ip, src, dest, clientip, client_ip, remote_addr, remote_ip, and anything matching *_ip or ip_*.

Field reference

All names assume the default speculus_ prefix.

Match status

Scoring

Network and location

Infrastructure flags

Residential proxy detail

Populated when the address has been observed as part of a proxy network.

Threat detail

Populated when the address has threat observations.

Links

• Speculus Threat Intelligence on Splunkbase
• Announcement: the Splunk integration is live
• Speculus MMDB

Questions? Email us at [email protected].