Skip to main content

I’ve been trying to export a list of curated rules using the Chronicle API. I attempted both the legacy Backstory endpoint and the newer Chronicle API.

When using the Backstory endpoint, I receive a 403 error indicating that the Backstory API isn't enabled. However, I’m unable to locate the Backstory API in the SecOps GCP project - only the Chronicle API appears to be available which is already enabled.

On the other hand, when I use the Chronicle API and try to use the curatedrules.list method, I receive a 404 error stating that the endpoint doesn't exist. I'm confident the {parent} parameter is valid, as I’ve successfully used it to export custom rules from the same SecOps tenant.

I’m using a service account with keys and the official api-samples-python scripts to retrieve the rule list.

 

I was able to resolve the issue after debugging my code.


@Rene_Figueroa @vishnu_manu 

I’m new to Google chronicle and I have been using the v1alpha endpoint from below documentation to list all curated rules but it only returns 30 rules. The documentation does confirm support for page size upto 1000 and page token. However, I don’t receive a page token. 

https://cloud.google.com/chronicle/docs/reference/rest/v1alpha/projects.locations.instances.curatedRules/list

On the other hand when I attempt to use v1alpha endpoint to list curated rules based on below documentation I’m able to retrieve all the ruleset name and I do see receive the next page token when it hit the limit.

https://cloud.google.com/chronicle/docs/reference/rest/v1alpha/projects.locations.instances.curatedRuleSetCategories.curatedRuleSets/list

Could you please help me understand why only 30 rules are being retrieved when using curated list endpoint.  


Hello @sree_k , it seems your code might be fetching data only from the first page.

To retrieve all the data you need, you may need to modify your code to handle pagination.

If you share your code, I'd be happy to help identify where adjustments can be made to fetch the full results.


I was able to resolve the issue after debugging my code.


1984 is my prefered code

Hello @sree_k , it seems your code might be fetching data only from the first page.

To retrieve all the data you need, you may need to modify your code to handle pagination.

If you share your code, I'd be happy to help identify where adjustments can be made to fetch the full results.


@vishnu_manu Thank you for your response.

I used the python code below and adjusted the scope and the endpoint. The code worked when I was using with the CuratedRuleset.list endpoint and it returned the next page token. So I’m not certain if it’s a pagination issue. I will look forward for the feedback from you.

https://github.com/chronicle/api-samples-python/blob/master/detect/v2/list_curated_rules.py


@vishnu_manu Thank you for your response.

I used the python code below and adjusted the scope and the endpoint. The code worked when I was using with the CuratedRuleset.list endpoint and it returned the next page token. So I’m not certain if it’s a pagination issue. I will look forward for the feedback from you.

https://github.com/chronicle/api-samples-python/blob/master/detect/v2/list_curated_rules.py


@sree_k ,
It does seem like pagination Issue,
Try this code and let me know if it works. 
https://github.com/Vishnu-Freak/Community/blob/main/list_curated_rules.py


@sree_k ,
It does seem like pagination Issue,
Try this code and let me know if it works. 
https://github.com/Vishnu-Freak/Community/blob/main/list_curated_rules.py


@vishnu_manu I tried your code with below modifications and it still retrieves the same 30 rule. 

_chronicle_api_base_url = "https://chronicle.googleapis.com"

SCOPES = [
"https://www.googleapis.com/auth/cloud-platform",
]
.....
.....

url = f"{_chronicle_api_base_url}/v1alpha/projects/{my_proj_id}/locations/{my_location}/instances/{my_instance_id}/curatedRules"
....
....
session = chronicle_auth.initialize_http_session(args.credentials_file,SCOPES)

The json output include the below message.
...../curatedRules with params: {'page_size': 1000}
DEBUG: Fetched 30 rules. Next page token:
Finished fetching all pages. Total rules: 30


@vishnu_manu I tried your code with below modifications and it still retrieves the same 30 rule. 

_chronicle_api_base_url = "https://chronicle.googleapis.com"

SCOPES = [
"https://www.googleapis.com/auth/cloud-platform",
]
.....
.....

url = f"{_chronicle_api_base_url}/v1alpha/projects/{my_proj_id}/locations/{my_location}/instances/{my_instance_id}/curatedRules"
....
....
session = chronicle_auth.initialize_http_session(args.credentials_file,SCOPES)

The json output include the below message.
...../curatedRules with params: {'page_size': 1000}
DEBUG: Fetched 30 rules. Next page token:
Finished fetching all pages. Total rules: 30


@vishnu_manu Additionally, as mentioned earlier I wasn't receiving a nextPageToken using my script while its the same with yours when page size is set to 1000. 

I tried to run the script by setting the page size to 1 and it appears that the each rule is one page and the script terminates exactly after page 30 without a nextpage token. 

Fetching page 29...
DEBUG: Requesting URL: https://europe-chronicle.googleapis.com/v1alpha/projects/{my_proj_id}/locations/{my_location}/instances/{my_instance_id}/curatedRules with params: {'page_size': 1, 'page_token': 'CgsImMH-redacted'}
DEBUG: Fetched 1 rules. Next page token: CgsImMH-redacted
Fetching page 30...
DEBUG: Requesting URL: https://europe-chronicle.googleapis.com/v1alpha/{my_proj_id}/locations/{my_location}/instances/{my_instance_id}/curatedRules with params: {'page_size': 1, 'page_token': 'CgsImMH-redacted'}
DEBUG: Fetched 1 rules. Next page token:
Finished fetching all pages. Total rules: 30


Reply