What Is Splunk – A Deep Dive

Splunk is not a tool, it is a platform on which tools are built. When is Splunk is implemented correctly it completely changes the business processes in place. If day to day activities are not changed after Splunk is in place then it was a failure. One of the many ways that Splunk can change a business is in the security landscape. Let’s discuss some of the ways that a single Splunk user can fundamentally change an institutions security posture.

Greater visibility

As any SANS instructor will tell you, you cannot protect what you can’t see. Understanding the network is core to protecting it. One should know what devices are on the network and the common patterns of communication. By deploying forwarders to servers you start to see patterns emerge and utilizing splints analytics you can start to look for outliers or patterns that resemble malicious behavior.

Alerts

Many institutions struggle with staffing. The cybersecurity world is a tough job market for people trying to recruit talent. There is not enough supply and far too much demand. A good thing if you are looking for a job, a bad thing if you are trying to fill a position. Often departments will only have 1 person or less to fill the role of security. Utilizing Splunk’s advance alerting options allow you to customize it to alert you when something relevant to your network occurs. Maybe a service account is being utilized in a suspicious way. Maybe Stephen is logging on at odd times. The possibilities are endless. As I stated before Splunk is a platform. If you are young enough to have played Minecraft or maybe have kids that do Splunk is like Minecraft or legos, limited only by your imagination.

In-depth analysis

Once you have your forwarders and an understanding of your traffic you can begin to utilize Splunk to analyze vents. Often you can use standalone Splunk instance to analyze data related to a possible breach or incident. Pull out all the relevant fields and data and feed it in a closed Splunk environment to analyze it. This will allow you to protect the data, segregate it from your standard operation environment, and customize your dashboards to investigate what happened.

Splunk can change everything you do on a day to day basis. You should be utilizing Splunk to automate all of your reports and your repetitive functions. The ability to recover time from your day to day work schedule is one of the best ROI of Splunk. This is especially crucial in a world with limited resources and increased threat. This does come with some trade-offs. This would not be a security blog if I didn’t at least warn you on some of the ways that you should be protecting Splunk.

Below is a small list of areas you should be concerned or hardening:

  • Heavy Forwarders- heavy forwarders are full Splunk instances and should be limited unless necessary. Be careful about where this is deployed.
  • Deployment servers- Deployment servers are really helpful and really powerful. Limit the amount access to it, make sure the forwarders are not being run as admin. Anybody that gains access to the forwarder can deploy code as the user that forwarders are run on.
  • Remove all web services from all Splunk instant save for search heads and the necessary hosts.
  • Run Splunk as not privileged on all hosts.
  • Utilize role-based access for data indexes.

Splunk provides documentation for hardening it. I would highly recommend following the documentation. Finally, look at GitHub and splunkbase for inspiration and tips on how to maximize your Splunk utilization. There are some amazing things being done in the operation and security world with Splunk. It can be the best platform for your day to day actives and maximize your time.

Happy Splunking and please contact us if you want help!

Cody Jackson

Security Architect at Aquila
Cody Jackson is a leader in the management and design of secure wired and wireless networks. He performs engineering assessments for a variety of commercial solutions. He helps CISOs, IT Specialists, and Security Analysts secure network and enterprise data.

Cody's engineering networking and security expertise broaden AQUILA’s ability to service organizations. Before AQUILA, Cody was a Computer and Information Security Specialist on the Los Alamos National Laboratory Security Architecture and Application Development team. He also worked as an IT Specialist at Western New Mexico University.

Cody earned a Master of Science in Information Systems & Assurance from UNM. He received an MBA and a B.S. in History and Math from Western New Mexico University.
Cody Jackson

Latest posts by Cody Jackson (see all)