Google SecOps Curated Rule export

I was able to resolve the issue after debugging my code.

@Rene_Figueroa @vishnu_manu 

I’m new to Google chronicle and I have been using the v1alpha endpoint from below documentation to list all curated rules but it only returns 30 rules. The documentation does confirm support for page size upto 1000 and page token. However, I don’t receive a page token. 

https://cloud.google.com/chronicle/docs/reference/rest/v1alpha/projects.locations.instances.curatedRules/list

On the other hand when I attempt to use v1alpha endpoint to list curated rules based on below documentation I’m able to retrieve all the ruleset name and I do see receive the next page token when it hit the limit.

https://cloud.google.com/chronicle/docs/reference/rest/v1alpha/projects.locations.instances.curatedRuleSetCategories.curatedRuleSets/list

Could you please help me understand why only 30 rules are being retrieved when using curated list endpoint.  

Hello @sree_k , it seems your code might be fetching data only from the first page.

To retrieve all the data you need, you may need to modify your code to handle pagination.

If you share your code, I'd be happy to help identify where adjustments can be made to fetch the full results.

@vishnu_manu Thank you for your response.

I used the python code below and adjusted the scope and the endpoint. The code worked when I was using with the CuratedRuleset.list endpoint and it returned the next page token. So I’m not certain if it’s a pagination issue. I will look forward for the feedback from you.

https://github.com/chronicle/api-samples-python/blob/master/detect/v2/list_curated_rules.py

@sree_k ,
It does seem like pagination Issue,
Try this code and let me know if it works. 
https://github.com/Vishnu-Freak/Community/blob/main/list_curated_rules.py

@vishnu_manu I tried your code with below modifications and it still retrieves the same 30 rule. 

_chronicle_api_base_url = "https://chronicle.googleapis.com"

SCOPES = [
"https://www.googleapis.com/auth/cloud-platform",
]
.....
.....

url = f"{_chronicle_api_base_url}/v1alpha/projects/{my_proj_id}/locations/{my_location}/instances/{my_instance_id}/curatedRules"
....
....
session = chronicle_auth.initialize_http_session(args.credentials_file,SCOPES)

The json output include the below message.
...../curatedRules with params: {'page_size': 1000}
DEBUG: Fetched 30 rules. Next page token:
Finished fetching all pages. Total rules: 30

@vishnu_manu Additionally, as mentioned earlier I wasn't receiving a nextPageToken using my script while its the same with yours when page size is set to 1000. 

I tried to run the script by setting the page size to 1 and it appears that the each rule is one page and the script terminates exactly after page 30 without a nextpage token. 

Fetching page 29...
DEBUG: Requesting URL: https://europe-chronicle.googleapis.com/v1alpha/projects/{my_proj_id}/locations/{my_location}/instances/{my_instance_id}/curatedRules with params: {'page_size': 1, 'page_token': 'CgsImMH-redacted'}
DEBUG: Fetched 1 rules. Next page token: CgsImMH-redacted
Fetching page 30...
DEBUG: Requesting URL: https://europe-chronicle.googleapis.com/v1alpha/{my_proj_id}/locations/{my_location}/instances/{my_instance_id}/curatedRules with params: {'page_size': 1, 'page_token': 'CgsImMH-redacted'}
DEBUG: Fetched 1 rules. Next page token:
Finished fetching all pages. Total rules: 30