Hi @keso. In addition to the methods you mentioned in your post I also wanted to highlight that it's possible to handle MISP log Ingestion through ingestion scripts that we provide on our GitHub repository.
These ingestion scripts are used with Cloud Run functions to accomplish ingestion and you can see the full list of supported sources and associated scripts in the repository.
https://cloud.google.com/chronicle/docs/ingestion/ingest-using-cloud-functions
https://github.com/chronicle/ingestion-scripts
Hi @keso. In addition to the methods you mentioned in your post I also wanted to highlight that it's possible to handle MISP log Ingestion through ingestion scripts that we provide on our GitHub repository.
These ingestion scripts are used with Cloud Run functions to accomplish ingestion and you can see the full list of supported sources and associated scripts in the repository.
https://cloud.google.com/chronicle/docs/ingestion/ingest-using-cloud-functions
https://github.com/chronicle/ingestion-scripts
Hi @vaskenh I followed the document and tried to run the scripts in cloud function. However am facing the below error
[builder] === Utils - Label Image (google.utils.label-image@0.0.2) ===
#############################################################
[12:47:42 AM] - Function is ready to test
#############################################################
[12:47:44 AM] - Traceback (most recent call last):
File "/layers/google.python.pip/pip/bin/functions-framework", line 8, in <module>
sys.exit(_cli())
^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
[12:47:44 AM] - return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/_cli.py", line 36, in _cli
app = create_app(target, source, signature_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/__init__.py", line 395, in create_app
raise e from None
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/__init__.py", line 376, in create_app
spec.loader.exec_module(source_module)
[12:47:44 AM] - File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/workspace/main.py", line 22, in <module>
from common import ingest
File "/workspace/common/ingest.py", line 29, in <module>
CUSTOMER_ID = utils.get_env_var(env_constants.ENV_CHRONICLE_CUSTOMER_ID)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/common/utils.py", line 51, in get_env_var
raise RuntimeError(f"Environment variable name is required.")
RuntimeError: Environment variable name is required.
Have you ever seen something like this ?
Hi @vaskenh I followed the document and tried to run the scripts in cloud function. However am facing the below error
[builder] === Utils - Label Image (google.utils.label-image@0.0.2) ===
#############################################################
[12:47:42 AM] - Function is ready to test
#############################################################
[12:47:44 AM] - Traceback (most recent call last):
File "/layers/google.python.pip/pip/bin/functions-framework", line 8, in <module>
sys.exit(_cli())
^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
[12:47:44 AM] - return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/_cli.py", line 36, in _cli
app = create_app(target, source, signature_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/__init__.py", line 395, in create_app
raise e from None
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/__init__.py", line 376, in create_app
spec.loader.exec_module(source_module)
[12:47:44 AM] - File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/workspace/main.py", line 22, in <module>
from common import ingest
File "/workspace/common/ingest.py", line 29, in <module>
CUSTOMER_ID = utils.get_env_var(env_constants.ENV_CHRONICLE_CUSTOMER_ID)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/common/utils.py", line 51, in get_env_var
raise RuntimeError(f"Environment variable name is required.")
RuntimeError: Environment variable name is required.
Have you ever seen something like this ?
It looks like the environment variable for your customer ID has not been set. Did you create an env.yml file and specify the file in the gcloud command deploying the function? If you are building the function in the UI, you can also fill these out in the runtime variables section.
https://cloud.google.com/functions/docs/configuring/env-var
It looks like the environment variable for your customer ID has not been set. Did you create an env.yml file and specify the file in the gcloud command deploying the function? If you are building the function in the UI, you can also fill these out in the runtime variables section.
https://cloud.google.com/functions/docs/configuring/env-var
Hi @cmorris I did tried everything as per the doc but still am facing the same error also I checked all the env.yml it's up to date with all required information
Hi @cmorris I did tried everything as per the doc but still am facing the same error also I checked all the env.yml it's up to date with all required information
Are you deploying via Cloud Shell or the UI? If Cloud Shell, can you share the (sanitized) command you are using?
Are you deploying via Cloud Shell or the UI? If Cloud Shell, can you share the (sanitized) command you are using?
Am using UI for the deployment
Am using UI for the deployment
Hi @cmorris In addition to this can we import the MISP IOC to GCS and the export them into Google Scopes using custom Python Script ?
Am using UI for the deployment
Can you try the following in place of the env.yml file as it looks like your env.yml file is not being used by the function:
Open the Functions Overview page in the Google Cloud console:
Click an existing function to go to its details page.
Click Edit.
Open the Runtime, build and connections settings section.
Select the Runtime tab.
Add in your Runtime environment variables.
Can you try the following in place of the env.yml file as it looks like your env.yml file is not being used by the function:
Open the Functions Overview page in the Google Cloud console:
Click an existing function to go to its details page.
Click Edit.
Open the Runtime, build and connections settings section.
Select the Runtime tab.
Add in your Runtime environment variables.
I did and still and am still facing the same error as mentioned previously