Skip to main content

Needhelp with yara rule to find the users who are accessing the website which is hosted in AppEngine

  • Apr 10, 2024 12:16 PM UTC
  • 4 replies
  • 189 views

Forum|alt.badge.img+1

Hi everyone,

I'm currently working on setting up some security monitoring for my Google App Engine-hosted website, and I'm looking to create a YARA rule in chronicle to detect users who are accessing the website. I'd like to monitor this activity for security and compliance purposes.

I tried various udm fields (target.) to find the users who accessing the website, But could'nt be able to find the correct UDM field.Could someone please provide guidance on how I can create an effective YARA rule to achieve this? Specifically, I'm looking to detect HTTPS/HTTP requests or any other relevant activity indicating user access to the website.

Any advice, examples, or resources would be greatly appreciated.

Thanks in advance!
Nikhil

4 replies

Ben_T
Staff
Forum|alt.badge.img+5
  • Staff
  • Apr 10, 2024 2:22 PM UTC

Hi NikhilBattula,

Here are some sample UDM searches applicable to Google App Engine default logging to help get you started following by a YARA sample:

  • Event Type: HTTP_SERVER_LOG (or whatever your equivalent log source is)
  • App Engine Domain: We used a placeholder "your-app-engine-domain.com"
  • HTTP Methods: GET and POST
  • Success Status Codes: 200 to 399

Relevant UDM Fields

Here are the most vital UDM fields you'd likely use for searching:

  • target.domain: The domain name the request was made to.
  • http_event.request.method: HTTP method (GET, POST, etc.)
  • http_event.response.status_code: The HTTP status code of the response
  • principal.ip: Client IP address making the request
  • principal.geoip.country: (If available) Country of origin based on IP address

Example UDM Searches

  1. Basic Access to Your Domain:

    event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" 
    
  2. POST Requests Only:

    event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" AND http_event.request.method = "POST"
    
  3. Successful Requests from a Specific Country:

    event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" AND http_event.response.status_code >= 200 AND http_event.response.status_code < 400 AND principal.geoip.country = "US" 
    

Important Notes:

  • Log Content: These searches assume your web server logs are being ingested into Chronicle and parsed to populate the correct UDM fields.
  • UDM Search Syntax: Remember that Chronicle supports basic operators like =, !=, >, <, AND, OR, etc. See the official docs for details (https://cloud.google.com/chronicle/docs/investigation/udm-search)
  • Refinement: UDM searches can get quite complex. Add more fields to filter, use regular expressions for sophisticated pattern matching, and leverage Chronicle's built-in functions if needed.

Pro Tip:

Start with simple searches, examine the results, then add more conditions iteratively to fine-tune your queries. Chronicle's search interface will offer autocomplete suggestions based on the available UDM fields in your ingested data.

Focus Points for the YARA Rule

  • IP Addresses: Client IP addresses will be the primary way to track 'users' in this scenario. Understand that a single IP address can represent multiple users over time (DHCP, shared devices, etc.).
  • Request Patterns: Target HTTP methods (GET, POST), requested URLs, user-agents, and potentially referrers. These help distinguish regular access from potentially malicious probes.
  • Geolocation: Consider IP-based geolocation to detect accesses from unusual regions.
  • Behavioral Baselines (If Possible): This depends on your logging depth. You might track regular access times/patterns per IP to detect anomalies.

##Example YARA-L rule

rule detect_website_activity {
meta:
author = "Your Name"
description = "Detects accesses to Google App Engine Website"
yara_version = "YL2.0"
rule_version = "1.0"

events:
$http_event.metadata.event_type = "HTTP_SERVER_LOG" // Adjust if your log type is different

condition:
$http_event.target.domain == "your-app-engine-domain.com" and
($http_event.request.method == "GET" or $http_event.request.method == "POST") and
$http_event.response.status_code >= 200 and
$http_event.response.status_code < 400 // Focus on successful requests
}

Explanation

  • meta: Descriptive information about the rule
  • events: Specifies the type of log events this rule applies to. Adjust 'HTTP_SERVER_LOG' if needed.
  • condition: The heart of the detection. We look for:
    • Traffic to your App Engine domain
    • GET or POST requests (common for browsing and logins)
    • Status codes indicating successful page loads/actions

Important Notes:

  1. Customize: Replace "your-app-engine-domain.com" with your actual domain. Add more specific URL rules if you like (e.g., targeting a login page).
  2. Log Availability: This rule ASSUMES you are feeding appropriate web server logs into Chronicle. If not, you must first set up that data flow.
  3. Refinement: This is a starting point. Enhance it over time by adding:
    • Geolocation checks
    • User-agent pattern matching (unusual browsers, etc.)
    • Anomaly detection (if you can baseline normal behavior)

Forum|alt.badge.img+1
  • Author
  • Bronze 1
  • Apr 11, 2024 10:07 PM UTC

Hi NikhilBattula,

Here are some sample UDM searches applicable to Google App Engine default logging to help get you started following by a YARA sample:

  • Event Type: HTTP_SERVER_LOG (or whatever your equivalent log source is)
  • App Engine Domain: We used a placeholder "your-app-engine-domain.com"
  • HTTP Methods: GET and POST
  • Success Status Codes: 200 to 399

Relevant UDM Fields

Here are the most vital UDM fields you'd likely use for searching:

  • target.domain: The domain name the request was made to.
  • http_event.request.method: HTTP method (GET, POST, etc.)
  • http_event.response.status_code: The HTTP status code of the response
  • principal.ip: Client IP address making the request
  • principal.geoip.country: (If available) Country of origin based on IP address

Example UDM Searches

  1. Basic Access to Your Domain:

    event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" 
    
  2. POST Requests Only:

    event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" AND http_event.request.method = "POST"
    
  3. Successful Requests from a Specific Country:

    event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" AND http_event.response.status_code >= 200 AND http_event.response.status_code < 400 AND principal.geoip.country = "US" 
    

Important Notes:

  • Log Content: These searches assume your web server logs are being ingested into Chronicle and parsed to populate the correct UDM fields.
  • UDM Search Syntax: Remember that Chronicle supports basic operators like =, !=, >, <, AND, OR, etc. See the official docs for details (https://cloud.google.com/chronicle/docs/investigation/udm-search)
  • Refinement: UDM searches can get quite complex. Add more fields to filter, use regular expressions for sophisticated pattern matching, and leverage Chronicle's built-in functions if needed.

Pro Tip:

Start with simple searches, examine the results, then add more conditions iteratively to fine-tune your queries. Chronicle's search interface will offer autocomplete suggestions based on the available UDM fields in your ingested data.

Focus Points for the YARA Rule

  • IP Addresses: Client IP addresses will be the primary way to track 'users' in this scenario. Understand that a single IP address can represent multiple users over time (DHCP, shared devices, etc.).
  • Request Patterns: Target HTTP methods (GET, POST), requested URLs, user-agents, and potentially referrers. These help distinguish regular access from potentially malicious probes.
  • Geolocation: Consider IP-based geolocation to detect accesses from unusual regions.
  • Behavioral Baselines (If Possible): This depends on your logging depth. You might track regular access times/patterns per IP to detect anomalies.

##Example YARA-L rule

rule detect_website_activity {
meta:
author = "Your Name"
description = "Detects accesses to Google App Engine Website"
yara_version = "YL2.0"
rule_version = "1.0"

events:
$http_event.metadata.event_type = "HTTP_SERVER_LOG" // Adjust if your log type is different

condition:
$http_event.target.domain == "your-app-engine-domain.com" and
($http_event.request.method == "GET" or $http_event.request.method == "POST") and
$http_event.response.status_code >= 200 and
$http_event.response.status_code < 400 // Focus on successful requests
}

Explanation

  • meta: Descriptive information about the rule
  • events: Specifies the type of log events this rule applies to. Adjust 'HTTP_SERVER_LOG' if needed.
  • condition: The heart of the detection. We look for:
    • Traffic to your App Engine domain
    • GET or POST requests (common for browsing and logins)
    • Status codes indicating successful page loads/actions

Important Notes:

  1. Customize: Replace "your-app-engine-domain.com" with your actual domain. Add more specific URL rules if you like (e.g., targeting a login page).
  2. Log Availability: This rule ASSUMES you are feeding appropriate web server logs into Chronicle. If not, you must first set up that data flow.
  3. Refinement: This is a starting point. Enhance it over time by adding:
    • Geolocation checks
    • User-agent pattern matching (unusual browsers, etc.)
    • Anomaly detection (if you can baseline normal behavior)

Hi Ben,

Thank you for guiding me and giving me an example yara rule. 

I tried to find the event type for app engine logs (I didn't see any UDM events (metadata.event_type) for App engine) and tried using target.url UDM Field also. but i culdn't be able to find it.I can be able to see only GCP_CLOUDAUDIT logs in chronicle. But i want to create rules (And alerts ) in chronicle based on (request_logs) type.googleapis.com/google.appengine.logging.v1.RequestLog logs (which we can see in logs explorer) of the app engine.

I am new to chronicle and using direct ingestion to send app engine logs into Chronicle, could you please share or guide me on how to send the app engine request logs to chronicle instance?

Thanks,
Nikhil


Forum|alt.badge.img+6
  • Bronze 2
  • Apr 12, 2024 10:08 AM UTC

Hi Nikhil

It looks like theres the following default log type and parser:

 

GCP_APP_ENGINE

 



I'd try to get the app engine logs ingested through that parser as a first step. 
How are you currently ingesting your data? 

EDIT for clarity: When you say direct ingestion could you clarify please


amandasamop
Forum|alt.badge.img+1
  • New Member
  • Sep 29, 2026 3:34 PM UTC

Hi NikhilBattula,

Here are some sample UDM searches applicable to Google App Engine default logging to help get you started following by a YARA sample:

  • Event Type: HTTP_SERVER_LOG (or whatever your equivalent log source is)
  • App Engine Domain: We used a placeholder "your-app-engine-domain.com"
  • HTTP Methods: GET and POST
  • Success Status Codes: 200 to 399

Relevant UDM Fields

Here are the most vital UDM fields you'd likely use for searching:

  • target.domain: The domain name the request was made to.
  • http_event.request.method: HTTP method (GET, POST, etc.)
  • http_event.response.status_code: The HTTP status code of the response
  • principal.ip: Client IP address making the request
  • principal.geoip.country: (If available) Country of origin based on IP address

Example UDM Searches

  1. Basic Access to Your Domain:

    event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" 
  2. POST Requests Only:

    event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" AND http_event.request.method = "POST"
  3. Successful Requests from a Specific Country:

    event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" AND http_event.response.status_code >= 200 AND http_event.response.status_code < 400 AND principal.geoip.country = "US" 

Important Notes:

  • Log Content: These searches assume your web server logs are being ingested into Chronicle and parsed to populate the correct UDM fields.
  • UDM Search Syntax: Remember that Chronicle supports basic operators like =, !=, >, <, AND, OR, etc. See the official docs for details (https://cloud.google.com/chronicle/docs/investigation/udm-search)
  • Refinement: UDM searches can get quite complex. Add more fields to filter, use regular expressions for sophisticated pattern matching, and leverage Chronicle's built-in functions if needed.

Pro Tip:

Start with simple searches, examine the results, then add more conditions iteratively to fine-tune your queries. Chronicle's search interface will offer autocomplete suggestions based on the available UDM fields in your ingested data.

Focus Points for the YARA Rule

  • IP Addresses: Client IP addresses will be the primary way to track 'users' in this scenario. Understand that a single IP address can represent multiple users over time (DHCP, shared devices, etc.).
  • Request Patterns: Target HTTP methods (GET, POST), requested URLs, user-agents, and potentially referrers. These help distinguish regular access from potentially malicious probes.
  • Geolocation: Consider IP-based geolocation to detect accesses from unusual regions.
  • Behavioral Baselines (If Possible): This depends on your logging depth. You might track regular access times/patterns per IP to detect anomalies.

##Example YARA-L rule

rule detect_website_activity {
meta:
author = "Your Name"
description = "Detects accesses to Google App Engine Website"
yara_version = "YL2.0"
rule_version = "1.0"

events:
$http_event.metadata.event_type = "HTTP_SERVER_LOG" // Adjust if your log type is different

condition:
$http_event.target.domain == "your-app-engine-domain.com" and
($http_event.request.method == "GET" or $http_event.request.method == "POST") and
$http_event.response.status_code >= 200 and
$http_event.response.status_code < 400 // Focus on successful requests
}

Explanation

  • meta: Descriptive information about the rule
  • events: Specifies the type of log events this rule applies to. Adjust 'HTTP_SERVER_LOG' if needed.
  • condition: The heart of the detection. We look for:
    • Traffic to your App Engine domain
    • GET or POST requests (common for browsing and logins)
    • Status codes indicating successful page loads/actions

Important Notes:

  1. Customize: Replace "your-app-engine-domain.com" with your actual website domain. Add more specific URL rules if you like (e.g., targeting a login page).
  2. Log Availability: This rule ASSUMES you are feeding appropriate web server logs into Chronicle. If not, you must first set up that data flow.
  3. Refinement: This is a starting point. Enhance it over time by adding:
    • Geolocation checks
    • User-agent pattern matching (unusual browsers, etc.)
    • Anomaly detection (if you can baseline normal behavior)

Hi Ben,

Thank you for guiding me and giving me an example yara rule. 

I tried to find the event type for app engine logs (I didn't see any UDM events (metadata.event_type) for App engine) and tried using target.url UDM Field also. but i culdn't be able to find it.I can be able to see only GCP_CLOUDAUDIT logs in chronicle. But i want to create rules (And alerts ) in chronicle based on (request_logs) type.googleapis.com/google.appengine.logging.v1.RequestLog logs (which we can see in logs explorer) of the app engine.

I am new to chronicle and using direct ingestion to send app engine logs into Chronicle, could you please share or guide me on how to send the app engine request logs to chronicle instance?

Thanks,
Nikhil

I also have the same issue, especially when trying to figure out which UDM fields are actually populated for web requests coming from an App Engine application. It can be confusing because the information you need may not necessarily appear under the target fields, depending on how the App Engine logs are being collected, normalized, and mapped into Chronicle UDM. I think the first thing to establish is what the raw App Engine request logs look like before they are normalized, because things such as source IP, hostname, URL, HTTP method, user identity, and request metadata may end up in different UDM fields. A YARA-L rule can only reliably match what is actually present in the normalized event, so identifying the correct event type and field mapping seems important before writing the detection logic.