Enabling ModSecurity in the Kubernetes Ingress-NGINX Controller

The Kubernetes Ingress-Nginx controller allows us to enable ModSecurity, an OpenSource Web Application Firewall. In this post I will go over how to use and configure ModSecurity with Ingress-Nginx.

“Marshmello walking down the side of the road with camera on the top of the building” by arvin febry on Unsplash

What is ModSecurity

In Ingress-NGINX you will be able to ModSecurity in a variety of ways:

  • Default Configuration
  • OWASP Core Rule Set
  • Snippet

Default Configuration

Usually you would run in “Detection Only Mode” for some time and review all the events generated by ModSecurity. Then you should start fine tuning it towards your specific application.

The default Configuration also contains different rules for ModSecurity which are heavily documented which you can review in the link above. It is recommended that you modify this file as needed.

OWASP Core Rule Set

It protects against the OWASP Top 10 and many others. Some of the common attacks it protects against are:

  • SQL Injection (SQLi): SQL Injection is an attack in which an SQL query is inserted into from client to application.
  • Cross Site Scripting (XSS): Cross-site scripting attacks occur when an attacker uses a web application to send malicious code to a different end user.
  • Local File Inclusion (LFI): Local File Inclusion is the process of including files, that are already locally present on the server, through the exploiting of vulnerable inclusion procedures implemented in the application.
  • Remote File Inclusion (RFI): Remote File Inclusion is the process of including remote files through the exploiting of vulnerable inclusion procedures implemented in the application.
  • Remote Code Execution (RCE): Code Injection is the general term for attack types which consist of injecting code that is then interpreted/executed by the application.
  • Session Fixation: Session Fixation is an attack that permits an attacker to hijack a valid user session.
  • Scanner Detection: Scanners can be used by hackers for malicious reasons. If you aren’t expecting any scan on your system, it is useful to detect that it is happening.
  • Metadata/Error Leakages: Leaking of descriptive errors or sensitive information.

Note, that the OWASP Core Rule Set configuration is highly documented and it is recommended that it be tuned.

Snippet

I recommended once you have ModSecurity configuration tuned with all of your needs you use the ModSecurityConfig directive to point toward your configuration.

In order to add a custom ModSecurity configuration, you can either add all the required code to the Snippet(maybe hard to maintain) or build an Ingress-NGINX image containing your ModSecurity configuration. Search for “build modsecurity library” in the nginx build file for an example.

Note: Customizing and Building NGINX with Custom ModSecurity files is out of the scope of this blog post, but there is some documentation here.

Setting Up the Ingress Controller

Note that this guide was written using Minikube version 0.31 running Ingress-Nginx version 0.21. If you already have Minikube installed, you may need to update.

Deploying an Application with ModSecurity

$ echo "
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: meow
spec:
replicas: 2
selector:
matchLabels:
app: meow
template:
metadata:
labels:
app: meow
spec:
containers:
- name: meow
image: gcr.io/kubernetes-e2e-test-images/echoserver:2.1
ports:
- containerPort: 8080
" | kubectl apply -f -
# wait a min for the deployment to be created
$ kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE
meow 2 2 2 2
# you should have 2 pods running
$ kubectl get pods
NAME READY STATUS
meow-5557bc7c54-cw2ck 1/1 Running
meow-5557bc7c54-kfzm5 1/1 Running

gcr.io/kubernetes-e2e-test-images/echoserver:2.1 just responds with information about the request.

2. Expose our pods using Services. More info about exposure here

$ echo "
apiVersion: v1
kind: Service
metadata:
name: meow-svc
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: meow
" | kubectl apply -f -
# wait a min for the service to be created
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
meow-svc ClusterIP 10.107.78.24 <none> 80/TCP

This allows the meow pods, created with the deployment above to be accessed from within the cluster. That way we can access the meow pods from the nginx-ingress-controller pod.

3. Setup the Ingress Rules, enabling ModSecurity with a custom snippet

$ echo "
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/enable-modsecurity: \"true\"
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecRuleEngine On
SecRequestBodyAccess On
SecAuditEngine RelevantOnly
SecAuditLogParts ABIJDEFHZ
SecAuditLog /var/log/modsec_audit.log
SecRule REQUEST_HEADERS:User-Agent \"fern-scanner\" \"log,deny,id:107,status:403,msg:\'Fern Scanner Identified\'\"
name: meow-ingress
namespace: default
spec:
rules:
- http:
paths:
- backend:
serviceName: meow-svc
servicePort: 80
path: /meow
" | kubectl apply -f -
$ kubectl get ing meow-ingress
NAME HOSTS ADDRESS PORTS
meow-ingress * 10.0.2.15 80

This ingress resource enables ModSecurity with custom rules for testing purposes. You can read more about these directives and many others in the ModSecurity documentation.

Testing it out

$ minikube ip
192.168.99.100
# Here you can see the the meow application response
$ curl https://192.168.99.100/meow -k
...
Request Headers:
accept=*/*
host=192.168.99.100
user-agent=curl/7.54.0
...
# This path is forbidden because of the SecRule for blocking any request containing the header "user-agent: fern-scanner"
$ curl https://192.168.99.100/meow -k -H "user-agent: fern-scanner"
...
<center><h1>403 Forbidden</h1></center>
...

We can also take a look at the audit-logs:

# The AuditLog will contain only warnings and errors since the SecAuditEngine directive is set to RelevantOnly, It will also contain the parts specified in SecAuditLogParts$ kubectl get pods -n kube-system | grep nginx
nginx-ingress-controller-6766d5b6c7-9qvfm
$ kubectl exec -it -n kube-system nginx-ingress-controller-6766d5b6c7-9qvfm cat /var/log/modsec_audit.log
---kRdKBbup---A--
[10/Dec/2018:23:32:08 +0000] 154448472869.334297 192.168.99.1 50447 192.168.99.1 443
---kRdKBbup---B--
GET /meow HTTP/2.0
host: 192.168.99.100
accept: */*
user-agent: fern-scanner
...
---kRdKBbup---E--
<html>\x0d\x0a<head><title>403 Forbidden</title></head>\x0d\x0a<body>\x0d\x0a<center><h1>403 Forbidden</h1></center>\x0d\x0a<hr><center>nginx/1.15.6</center>\x0d\x0a</body>\x0d\x0a</html>\x0d\x0a
---kRdKBbup---F--
HTTP/2.0 403
Server: nginx/1.15.6
Date: Mon, 10 Dec 2018 23:32:08 GMT
Content-Length: 153
Content-Type: text/html
Connection: close
---kRdKBbup---H--
ModSecurity: Access denied with code 403 (phase 1). Matched "Operator `Rx' with parameter `fern-scanner' against variable `REQUEST_HEADERS:user-agent' (Value: `fern-scanner' ) [file "<<reference missing or not informed>>"] [line "7"] [id "107"] [rev ""] [msg "Fern Scanner Identified"] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "192.168.99.1"] [uri "/meow"] [unique_id "154448472869.334297"] [ref "o0,12v64,12"]
...

Troubleshooting

  1. Examine the Ingress-Nginx Logs:
# Note: I added "Ahhhh Ok" Somewhere in the ModSecurity Snippet that would cause an issue$ kubectl get pods -n kube-system | grep nginx
nginx-ingress-controller-6766d5b6c7-9qvfm
$ kubectl logs -n kube-system nginx-ingress-controller-6766d5b6c7-9qvfm
...
Error: exit status 1
2018/12/11 03:56:03 [notice] 7723#7723: ModSecurity-nginx v1.0.0
2018/12/11 03:56:03 [emerg] 7723#7723: "modsecurity_rules" directive Rules error. File: <<reference missing or not informed>>. Line: 3. Column: 12. Invalid input: Ahhhh Ok in /tmp/nginx-cfg303221100:331
nginx: [emerg] "modsecurity_rules" directive Rules error. File: <<reference missing or not informed>>. Line: 3. Column: 12. Invalid input: Ahhhh Ok in /tmp/nginx-cfg303221100:331
nginx: configuration file /tmp/nginx-cfg303221100 test failed
...

2. Examine the Nginx configuration which presented an error:

# Check the temporary config generated, around the line(331) stated in the error in step 1
$ kubectl exec -it -n kube-system nginx-ingress-controller-6766d5b6c7-9qvfm -- sed -n '325,335p' /tmp/nginx-cfg303221100
# After resolving the issue you can examine the configuration to see if your changes took effect
$ kubectl exec -it -n kube-system nginx-ingress-controller-6766d5b6c7-9qvfm cat /etc/nginx/nginx.conf
# Configuration checksum: 2601299549274808909
...

Annotations

  • nginx.ingress.kubernetes.io/enable-modsecurity: This must be set to true in order to enable ModSecurity. If enable-owasp-core-rules or a modsecurity-snippet annotations are not present, then ModSecurity will run in “Detection-Only” mode using the recommended configuration.
  • nginx.ingress.kubernetes.io/enable-owasp-core-rules: Enables the OWASP Core Rule Set. Note, it should not be used with the modsecurity-snippet annotation.
  • nginx.ingress.kubernetes.io/modsecurity-snippet: Allows you to add your own set of modsecurity rules via a snippet. Note, it should not be used with the enable-owasp-core-rules annotation.
  • nginx.ingress.kubernetes.io/modsecurity-transaction-id: Allows you to pass transactionIDs from nginx to ModSecurity.

For more detailed information on all the annotation usage see the official documentation.

Further Reading

The following are great resources to get started:

Thanks for Reading and I hope you enjoyed 😃

Senior Technical Marketing @ GitLab 🦊, Developer @ home, Keeping ATX Weird 😜

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store