Skip to main content

Hello community! there are a lot of posts about the MISP integration and also a lot of updates in SecOps. I wanted to know, to this date, how can I ingest MISP data to chronicle?  Would be enough following [1]? 

Is the BindPlane/Collection agent necessary to ingest the data? what is the difference between the collection agent and the MISP integration [2]? Could you summarise what is required to do the ingestion? 

Thank you!

 

[1] https://medium.com/@thatsiemguy/misp-bindplane-and-google-secops-262f48f9bdbd

[2] https://cloud.google.com/chronicle/docs/soar/marketplace-integrations/misp 

Hi @keso.  In addition to the methods you mentioned in your post I also wanted to highlight that it's possible to handle MISP log Ingestion through ingestion scripts that we provide on our GitHub repository.


These ingestion scripts are used with Cloud Run functions to accomplish ingestion and you can see the full list of supported sources and associated scripts in the repository. 


https://cloud.google.com/chronicle/docs/ingestion/ingest-using-cloud-functions


https://github.com/chronicle/ingestion-scripts


Additional information available: Configure MISP Integration to work with Google Security Operations SOAR


or review last response to https://www.googlecloudcommunity.com/gc/SOAR-Forum/Can-i-Integrate-Chronicle-SIEM-with-MISP-or-a-similar-platform/m-p/639179#M1122


Hi @keso.  In addition to the methods you mentioned in your post I also wanted to highlight that it's possible to handle MISP log Ingestion through ingestion scripts that we provide on our GitHub repository.


These ingestion scripts are used with Cloud Run functions to accomplish ingestion and you can see the full list of supported sources and associated scripts in the repository. 


https://cloud.google.com/chronicle/docs/ingestion/ingest-using-cloud-functions


https://github.com/chronicle/ingestion-scripts


Hi @vaskenh I followed the document and tried to run the scripts in cloud function. However am facing the below error 

[builder] === Utils - Label Image (google.utils.label-image@0.0.2) ===


#############################################################
[12:47:42 AM] - Function is ready to test
#############################################################


[12:47:44 AM] - Traceback (most recent call last):
File "/layers/google.python.pip/pip/bin/functions-framework", line 8, in <module>
sys.exit(_cli())
^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1157, in __call__

[12:47:44 AM] - return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/_cli.py", line 36, in _cli
app = create_app(target, source, signature_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/__init__.py", line 395, in create_app
raise e from None
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/__init__.py", line 376, in create_app
spec.loader.exec_module(source_module)

[12:47:44 AM] - File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/workspace/main.py", line 22, in <module>
from common import ingest
File "/workspace/common/ingest.py", line 29, in <module>
CUSTOMER_ID = utils.get_env_var(env_constants.ENV_CHRONICLE_CUSTOMER_ID)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/common/utils.py", line 51, in get_env_var
raise RuntimeError(f"Environment variable name is required.")
RuntimeError: Environment variable name is required.

Have you ever seen something like this ?


Hi @vaskenh I followed the document and tried to run the scripts in cloud function. However am facing the below error 

[builder] === Utils - Label Image (google.utils.label-image@0.0.2) ===


#############################################################
[12:47:42 AM] - Function is ready to test
#############################################################


[12:47:44 AM] - Traceback (most recent call last):
File "/layers/google.python.pip/pip/bin/functions-framework", line 8, in <module>
sys.exit(_cli())
^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1157, in __call__

[12:47:44 AM] - return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/_cli.py", line 36, in _cli
app = create_app(target, source, signature_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/__init__.py", line 395, in create_app
raise e from None
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/__init__.py", line 376, in create_app
spec.loader.exec_module(source_module)

[12:47:44 AM] - File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/workspace/main.py", line 22, in <module>
from common import ingest
File "/workspace/common/ingest.py", line 29, in <module>
CUSTOMER_ID = utils.get_env_var(env_constants.ENV_CHRONICLE_CUSTOMER_ID)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/common/utils.py", line 51, in get_env_var
raise RuntimeError(f"Environment variable name is required.")
RuntimeError: Environment variable name is required.

Have you ever seen something like this ?


It looks like the environment variable for your customer ID has not been set. Did you create an env.yml file and specify the file in the gcloud command deploying the function? If you are building the function in the UI, you can also fill these out in the runtime variables section.


https://cloud.google.com/functions/docs/configuring/env-var


It looks like the environment variable for your customer ID has not been set. Did you create an env.yml file and specify the file in the gcloud command deploying the function? If you are building the function in the UI, you can also fill these out in the runtime variables section.


https://cloud.google.com/functions/docs/configuring/env-var


Hi @cmorris I did tried everything as per the doc but still am facing the same error also I checked all the env.yml it's up to date with all required information 


Hi @cmorris I did tried everything as per the doc but still am facing the same error also I checked all the env.yml it's up to date with all required information 


Are you deploying via Cloud Shell or the UI? If Cloud Shell, can you share the (sanitized) command you are using?


Are you deploying via Cloud Shell or the UI? If Cloud Shell, can you share the (sanitized) command you are using?


Am using UI for the deployment


Am using UI for the deployment


Hi @cmorris In addition to this can we import the MISP IOC to GCS and the export them into Google Scopes using custom Python Script ?

 


Am using UI for the deployment


Can you try the following in place of the env.yml file as it looks like your env.yml file is not being used by the function:




  1. Open the Functions Overview page in the Google Cloud console:




  2. Click an existing function to go to its details page.




  3. Click Edit.




  4. Open the Runtime, build and connections settings section.




  5. Select the Runtime tab.




  6. Add in your Runtime environment variables.




Can you try the following in place of the env.yml file as it looks like your env.yml file is not being used by the function:




  1. Open the Functions Overview page in the Google Cloud console:




  2. Click an existing function to go to its details page.




  3. Click Edit.




  4. Open the Runtime, build and connections settings section.




  5. Select the Runtime tab.




  6. Add in your Runtime environment variables.




I did and still and am still facing the same error as mentioned previously 


Reply