Building a Splunk CIM compatible source addon

This walk through will build a Splunk CIM compatible source addon extending the CEF source type from my CEF framework TA. This is part three in a three part Series

Before you start, I will have to gloss over many topics you should have:

  • Read the prior two articles in this series.
  • You should also be comfortable with the ways Splunk can be used to parse and enrich data notable, TRANSFORMS, REPORT, EXTRACT, EVAL, and LOOKUP. A great cheat sheet is available from Alpura 
  • Be familiar with the web data model.
  • Be familiar with the data dictionary for our sample data

In the prior two articles we create a project style development environment for our add-on and a minimally viable set of field parse but have not yet considered any specific model. Reviewing our samples and vendor documentation we learn the data is most similar to a web access log which is known as the “web” model in the Splunk Common information model. In our case we have only two events available and vendor documentation that describes the events. When replicating this process in a new data source all unique events should be considered.

  • All events are a web access event
  • A subset of these events are “attack” events which do not have a web model to compare to.

Considering this we will use the “web” model as our basis and add a number of connivence elements for our users. The following table illustrates how we will map our data.

 

The implementation of the mapping is explained in the following table our implementation of the mapping can be viewed in bitbucket. Review default/props.conf default/transforms.conf and the lookups present in lookups/

CEF FieldSplunk FieldNotes
sipdest_ipNot a CIM field in the web model used by convention.
Not present in sample not validated
sptdest_portNot a CIM field in the web model used by convention.
Not present in sample not validated
qstruri_queryThe formatting of this field contains escaped equal (=) signs and is omitting the leading question mark (?) used a complex eval to adjust
cs9Rule_Name AND signatureNot a CIM field however Rule_Name is similar to an attack signature. Use an eval to split by comma and remove empties
Attack SeverityCEF_severity
severity
This field requires a lookup to set severity as one of low,medium,high,critical created “imperva_incapsula_severity.csv
requestmethodhttp_method
refhttp_referrer
requestClientApplicationhttp_user_agent
destsite
src_user_iduser
actionactThis field requires a lookup to translate act to action which can be allowed or blocked
vendor_productConstant string “Imperva Incapsula”
appvendor_appsaved for user search not used in the CIM model
appConstant String “incapsula”
actcachedSome values of act can indicate cached which is set to true using the actions lookup above

Creating Eventtypes

Fields alone are not enough to include an event in a datamodel. In-fact incorrect configuration of eventtypes and tags and include data which is invalid for a model compromising the usefulness of a model.

We will create two eventtypes for this data, our implementation can be viewed in eventype.conf using the bitbucket link above:

eventtypedescription
imperva_incapsula_webAll events matching our source and sourcetype
imperva_incapsula_web_attackAll eventtypes imperva_incapsula_web with a signature field.

Creating Tags

The final step to include events in a data model is to tag the events. Additional tags can be created in this case “attack” make sense for the subset of events that indicate a detection by the Incapsula WAF service. Tags which are not used by the data model are not included by default and are only available to the users in search activities.

tagdescription
webeventtype=imperva_incapsula_web
attackeventtype=imperva_incapsula_web_attack

Testing our work

Using “make package_test” ensure no unexpected errors or warnings are produced.

Using our development environment and EventGen via “make docker_dev” we can interactively validate our mapping is CIM compliant.

Leave a Reply