[Public Preview] Building Custom Integrations using local IDEs and managing them on Github

Hey folks,

As you all know, our built-in IDE, despite being a powerful tool, still has a lot of limitations, which can make the development process frustrating. Instead of trying to reinvent the wheel, we want to enable developers to work in the environments they are more used to and to be able to leverage all of the latest technologies to make this process as simple as possible.  

In this post, we would like to present a new flow, which enables you to use the local IDE environment to create/update integrations and manage it on Github.

In simple terms, this new approach consists of 2 steps:

• Make all of the changes in the forked version of our existing Github repository
• Pull/Push updated content to Google SecOps via our developed mp tool

Main benefits of this approach is that you fully own your version of Github repository. You can add dedicated CI/CD workflows, pre-commit hooks, AI integrations, Github actions  - anything to operate efficiently.

❗Note: this is still a Preview version  and we are actively working to ensure that everything works in a stable manner. If you encounter any issues, please share a comment under this post, so that everyone can see it and we can troubleshoot it together.❗

Understanding the new structure of Response Integrations

The structure of Response Integration was optimized for ease of development. 

Now, Response Integration consists of:

1. Metadata: YAML files describing the integration, its capabilities, parameters, etc.
2. Scripts: Python code that implements the integration functionality
3. Dependencies: Required Python packages for the integration

my_integration/

├── metadata/         # Metadata YAML files

│   ├── action.yaml   # Action definitions

│   ├── connector.yaml # Connector definitions

│   └── job.yaml      # Job definitions

├── scripts/          # Python implementation

│   ├── actions/      # Action implementations

│   ├── connectors/   # Connector implementations

│   └── jobs/         # Job implementations

├── tests/            # Test suite

└── pyproject.toml    # Python project configuration

Response Integration can exist in two states:

1. Non-built: Source code format used during development
2. Built: Packaged format ready for deployment to the Content Hub

For more information, visit this page.

 

First Steps

To get started in this new workflow, you need to do the following:

1. Clone content-hub repo
2. install mp cli tool
3. Run mp dev-env login command

For more information, visit this page.

How to pull existing changes from Google SecOps
 

Let’s assume that you already have custom integrations built in IDE. To make them accessible on Github, you will need to pull them by running:

mp dev-env pull integration <integration name from the SOAR IDE> –dst ’<absolute path>/response_integrations/custom’

This will download the integration from the Google SecOps instance into a “response_integrations/custom” folder and allow you to edit it in your local environment. The downloaded integration will be in the “non-built” state.

How to push changes to Google SecOps

To push the changes to custom integration to the Google SecOps environment, you need to run:
 

mp dev-env push integration <folder name, where integration is stored> –custom

This will automatically build the integration stored in the response_integrations/custom folder and push the updates to your Google SecOps environment. You should see all of the changes in the IDE ready for testing.

Note: if you store custom changes in a dedicated folder then you will need to run:

mp dev-env push integration <folder name, where integration is stored> --src <path>

Let us know, if you have any questions. This new approach will be considered the best practice and over time we plan to migrate all official integrations to be accessible on Github as well.

We are very excited and would like to hear any feedback from you!