Integrating Baseline Security Needs into your Company.

Posted on Sat 12 October 2019 in devsecops

The post tries to deal/note down points on some of my thoughs on baselining security into your organsiation and integrating security components for each phase of the stack. It is incomplete in many senses as there always tons of things you could integrate and solely depends where your company stands in terms of security and what tradeoff are they willing to do achieve the same, there is no one shot silver bullet strategy you could use out of the box. The post contains most of things i have worked on as well as some of the general observation and best practices i have taken,seen in some of the talks, papers etc.

Working in a team where the development and workflow is quite rapid, the importance to keep up security and offer the base minimal security for components is ideal and this was something focused on writing which structure the organisation for security.

When a single security engineer is being hired, the roles and responsibities should be upfront from the organisation and should have a frequent meeting with the team/member in order to build a culture of security as well as understand what could be improved and proper feedback taken and tasks to be decided best suited for the company.

A lot of the following details are written on the assumption made that the software is written and hosted at cloud.

The usual template followed during development and setting up of product for the organistaion is:

Organisational Requirement:

  • Get a Identity provider (GSuite, Microsoft)
  • Get a cloud provider (AWS. Google Cloud)

Devops/CI-CD Workflow:

  • Start Development of code. (Soruce) (Github, Gitlab)
  • Building code using a deployment software. (Build)
  • Testing the software (Test)
  • Deploying the artifact to required environments (QA, Dev, Staging, Prod) (Deployment.)
  • Continours Monitoring the deployment.

The best way you could integrate security in the code phase is to integrate into the pipeline and version control directly.

Security in Source (Version Control)

The ideal usecase for writing security integrated web application is to use a framework, for ex: django, flask etc which takes care of security and authentication and minimal effort is required there.

Out of the many ways to enforce security controls in the source, there are 2-3 ways which i think should be pretty good fpr baselining.

  • Integration of security as git hook.
  • Static code analysis
  • Writing security test cases for core modules like authentication, session timeouts, cookies and csrf. The rest of the test cases can be built as needed.
Integration as a git hook:

This step ensures that code has a basic sanity check, well this can be irritating for developers if the code base is high. A git hook can be modified to be suitable for only the specific commit.

There are variety of the things that can go into the git hook like

  • Secrets check (High entropy strings) - This can be done through detect-secrets.
  • Static code analsysis - Quite slow, if this is a huge codebase and can be cumbersome for the developers.
Static Code Analysis

Static Code analysis can be a tricky component to integrate depending on aa few factors:

  • The language you use and the corresponding tool used. (as far as i know python and java have some of the good opensource static code analysis tools, haven't got a chance to checkout checkmarx and other paid alternatives.)
  • The amount of false positives arising during integration phase, this is something inevitable and the corresponding team should have sufficient bandwidth to address the same. This is not ideal when the company is on-going rapid development like in a startup and has a sufficiently large code base. The ideal stage would be roll it out to parts of the code which you feel would be really ideal to integrate and work on it, and when stages reach where the security team is confident enough to roll out and there is sufficient bandwidth from the development team to address these issues.

Security in Building Code

This is the portion where the CI/CD deployment pipleine gets involved and you could have a flurry of tools installed. Things like

  • Docker Vulnerability scanner - Docker vulnerability scanners are required to ensure your images go vulnerability free. Clair Scanner is one of the most common docker scanner out there.

    clair-scanner --ip YOUR_LOCAL_IP alpine:3.5 --threshold Critical
    (This would ensure the build images would fail only if the vulnerability is a critical one.)

This could be supported by inhouse cloud provider like Google Cloud registry support your registry scanning for vulnerabilities.

  • Docker Images:- Maintaining a Base Image: It is best recommended to maintain a security verified hardened base image for the needs for the organsistaion and then inherit as per required.

  • Dockerfile Scanning: This is something, done to ensure no vulnerability pops up while writing these files. Scan for things like ADD doesnt fetch from links. Ensure the verified base image is being used in the file.

You could follow some of the best practices for docker mentioned here https://snyk.io/blog/10-docker-image-security-best-practices/

  • Dependency Scan - Dependency scan is used to scan and report any exisitng 3rd party dependencies in your project. There are many tools available for the same.

    Opensource - Owasp Dependency scan (Supports nodejs, python, java)
    (https://www.owasp.org/index.php/OWASP_Dependency_Check)

    Snyk (support java, python, go), details -> https://snyk.io/docs/

Testing the Product

This is where the QA testing of the product happens, the ideal use case would be to integrate the dast here.

If the product demands a very rapid development phase then the ideal usecase would be to scan the app for a baseline set of security. What would this cover ?

  • Ensure authentication works as expected.
  • CSRF tokens are being validated.
  • Security headers are in place.

OWASP ZAP baseline scanner is a good place to get started.

If the release has time or the testing could be done asnychronously then you could hook up ZAP or Burp suite to get the completed testing suite done. Most DAST kind of fail to identify issues on modern SPA due to application and logic flow as well.

REST API Scanning

One key thing to be noted is if you have a openapi or swagger rest api defined, its always better to run it against a REST API scanners like which can be used to find vulnerabilities like IDOR to a certain extent. * Astra - https://github.com/flipkart-incubator/Astra

GraphQL is onething that has come up quite frequently for organisations

Graphql endpoints like any other endpoints needs to be tested for common web application vulnerabilities. Make sure the schema and endpoints are well aware within the security team. Un-noted endpoints and often misconfigurations leads to exposing endpoints not required. ( DoyonSec has a nice tool for the pentesters https://github.com/doyensec/graph-ql)

Deployment of Product

Once all these phases integrate, it basically either a click of button in the pipeline to deploy or if all of the tests succeds it autodeploys.

A detailed set of toolset, you could integrate can be found here:

Image of Owasp

Cronjobs and Periodic Tests

This involves a basic list of checks, the organisation can keep as a cronjob or do it periodically.

Infrastructure Baseline Monitoring.

The ideal goal is here is to not secure the entire infrastructure, but ensure some of the key infrastructure components under no circumstances get exposed or exploited.

  • Monitor for any subdomain takeover (A k8 cronjob can be used for this, along with a wrapper around your dns managment provider - AWS/GCloud/Azure).
  • Check for expected ports open only on the product side.(80, 443 etc.)
  • Nginx/Apache hardened configs.
  • If ssh enabled on any publicly required instance (ensure pubkey auth and not password based), utilities like fail2ban can also help.
Web Application Pentests

This should be periodic, ideally every main component that goes into production, if needed to set a time frame probably every month. Follow the normal web application security guidlines for testing (OWASP, SANS 25)

Infrastructure Security

This part tries to cover the infrastructure security part and how it could help to detect attacks and have a threat detection system in place.

I dont know who said this but these are words for wisdom "If authentication and identity is taken care of, you have solved a lot of your internal security issues"

Managing Identity and Authentication in the organisation.

The dire advantage is quite visible any employee leaving or entering the organisation can be assigned the applications required in no time and can be removed as well during leaving. A startup should ideally be only using SSO based apps, this ensures that the identity is authenticated properly along with assosciated authorisation to internal and apps.

Some of the SSO providers are:

  • Okta
  • Google SSO

2FA for Employees.

Configure 2FA for employees, preferably yubikey if not TOTP and no SMS based 2 auth please. This is a real defense in depth in case of compromise.

This can be made a mandatory option in Gsuite or any other SSO you are using.

Cloud Security

A mix of the cloud provider security features as well as some of the inputs/automation from the security team would constitute the ideal cloud security model

Some of the companies would include, usecases to cover in cloud security be IAM/Service Accounts monitoring for employees as well as to see any unknown patters any unwanted permissions given to the employee etc.

Cloud security would also include monitoring for buckets/storage for providers, evaluating instance security within the company etc.

Platform Security Features

  • As for AWS, it has AWS Guardduty which does protect your AWS accounts and workloads with intelligent threat detection and continuous monitoring. AWS Guardduty doesn't offer any host based detections for the moment. But it does detect things like cryptomining etc from the network logs collected.
  • Google Cloud has the Google CLoud Security Center which is more or less the same as AWS Guardduty and has collective dashboard to it.
  • Azure has a security center itself, not awaare of all the features but is more or less the same.

There are lot of individual components which can be finetuned as well such as features in IAM/Google SA etc which allows to finetune policies and assosciate users within certain groups. This is all together a different topic and can be written as a separate topic.

Security team Automation

  • Hooks/Wrappers for Getting these relevant alerts to your messaging (slack) or corresponding channels for the team to respond.
  • Writing a lambda/serverless function or a cronjob to ensure Cloud Platform Security Feature (guardduty, cloud security center etc.) remain active at all times, if not enable it. Lets say an attacker gets control of a key, the first attempt might be to disable logging and guardduty.

Assess your current Security posture in the Cloud.

Once these features are enabled, this is where the security team pro-actively comes in, you could use tools like the cloud security scanner to evaluate the security components you might be missing out or mis-configured and take appropriate actions.

Some of the commonly used tools are:

  • Cloud security Suite
  • Scout Security Suite (https://github.com/nccgroup/ScoutSuite)

For security scanning both AWS and Gcloud have a security access role which can be used to create the keys correspondingly.

Monitoring and Alerts

This is a key part for an organisation to ensure the monitoring and alerts part of these integrated tools function properly. To succefully query out the metrics and to customise the alerts,it is ideal to integrate into one of the monitoring & alerting platform.

My knowledge within this is actually quite limited, i have been working on setting up a cluster and try to carve out some queries and filters which could be used to monitor some of the critical parts of the cloud like:

  • How many times was the IAM administrator account accessed.
  • Was it access from whitelisted set of IP's or not.
  • Are certain cloud API endpoints being accessed quite often than usual. etc.
  • Are certain containers spiking up than usual.

You could use any of the platforms like Datadog, Prometheus etc which actively support querying.

As for alerting, it could be passed on to your slack channels or corresponding team communication platform, raise a ticket for the issue automatically and assign it, if the tool is certain regarding the same.

Incident Response.

A well defined procedure must be documented, to follow in case of breach of a system, a user or the application etc. Some of the suggestions to start with would be:

  • Shutting down the instance or deleting the infected node.
  • Incase of a web application vulnerability, if developers are around roll out a fix for the issues asap. If not add a rule to block the specific endpoint for a while in the Load balancer rule settings.
  • Incase of credential compromise, block it. Always provide the developer with sufficient steps to contact the security team or concerned on-call or resource.
  • A security page must be published on the site to contact in case of a bug/security issue to be reported.

There are lot of things that can be added to the list, do come up as per the organsiation culture.

References

Some of the useful tips recieved are from:

  • How Leading Companies Are Scaling Their Security - Clint Gibler
  • Detecting Malicious Cloud Account Behavior A Look at the New Native Platform Capabilities - Brad Geesman
  • OWASP for pipeline tools: http://owasp.org/images/b/b5/DevOps_AppSec_Tool_Integration.png

This blog might have missed out on a numerous things as it was focussed on baselining, do let me your thoughts or feedbacks, i would be more than happy to add them in or correct anything is wrong.

Lets squash some bugs <3.

devsecops security in ci/cd development pipeline security baseline security for companies cloud security