Analyzing PFsense logs in Graylog3

Page content

We will parse the log records generated by the PfSense Firewall. We already have our graylog server running and we will start preparing the terrain to capture those logs records.

Many thanks to opc40772 developed the original contantpack for pfsense log agregation what I updated for the new Graylog3 and Elasticsearch 6.

Celebro localinstall

# celebro van to use port 9000 so change graylog3 bindport
nano /etc/graylog/server/server.conf
http_bind_address = 127.0.0.1:9400
nano /etc/nginx/conf.d/graylog.conf

systemctl restart graylog-server.service
systemctl restart nginx

wget https://github.com/lmenezes/cerebro/releases/download/v0.9.3/cerebro-0.9.3-1.noarch.rpm
yum localinstall cerebro-0.9.3-1.noarch.rpm

sudo sed -i 's|# JAVA_OPTS="-Dpidfile.path=/var/run/cerebro/play.pid"|JAVA_OPTS="-Dpidfile.path=/var/run/cerebro/play.pid -Dhttp.address=0.0.0.0"|' /etc/default/cerebro

chown cerebro:cerebro -R /usr/share/cerebro

systemctl start cerebro

Create indices

We now create the Pfsense indice on Graylog at System / Indexes image

Import index template for elasticsearch 6.x

systemctl stop graylog-server.service

git clone https://github.com/devopstales/pfsense-graylog.git
cd pfsense-graylog/service-names-port-numbers/
cp service-names-port-numbers.csv /etc/graylog/server/

Go to celebro > more > index templates Create new with name: pfsense-custom and copy the template from file pfsense_custom_template_es6.json

In Cerebro we stand on top of the pfsense index and unfold the options and select delete index.

Geoip database

wget -t0 -c http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz
tar -xvf GeoLite2-City.tar.gz
cp GeoLite2-City_*/GeoLite2-City.mmdb /etc/graylog/server

systemctl start graylog-server.service

Enable geoip database at System > Configurations > Plugins > Geo-Location Processor > update Chane the order of the Message Processors Configuration

  • AWS Instance Name Lookup
  • Message Filter Chain
  • Pipeline Processor
  • GeoIP Resolver

Enable geoip database

Import contantpack

import chaneg date timezone in Pipeline rule Go tu Stream menu and edit stream image

System > Pipelines Manage rules and then Edit rule (Change the timezone)

rule "timestamp_pfsense_for_grafana"
 when
 has_field("timestamp")
then
// the following date format assumes there's no time zone in the string
 let source_timestamp = parse_date(substring(to_string(now("Europe/Budapest")),0,23), "yyyy-MM-dd'T'HH:mm:ss.SSS");
 let dest_timestamp = format_date(source_timestamp,"yyyy-MM-dd HH:mm:ss");
 set_field("real_timestamp", dest_timestamp);
end

Confifure pfsense

Status > System Logs > Settings image

Confifure Opnsense

Access the Opnsense GUI System menu, access the Settings sub-menu and select the Logging / Targets option. image

Add a new logging target and perform the following configuration:

image

Install grafana Dashboard

# install nececery plugins
grafana-cli plugins install grafana-piechart-panel
grafana-cli plugins install grafana-worldmap-panel
grafana-cli plugins install savantly-heatmap-panel
systemctl restart grafana-server

Create new datasource:

image

Import dashboadr from store: id: 5420

image

Contantpack:

https://github.com/devopstales/pfsense-graylog