reCAPTCHA v3 Risk Score Assessment Becoming Inconsistent

Some ideas below for review: 

Experiencing sudden score drops or seeing genuine users receive low scores (down to 0.0–0.2) is a very common challenge during the rollout of reCAPTCHA v3 / Enterprise score-based keys.

Here are the detailed answers to your 5 questions, along with recommended best practices:

1. What could cause genuine transactions to start receiving significantly lower scores?

• Token Timing & Lack of Page Telemetry: reCAPTCHA v3 relies on behavioral telemetry (mouse movements, scrolling, typing cadence) gathered while the user interacts with the page. If grecaptcha.execute is triggered immediately on button click without preceding page interaction, or if the token is verified on your server more than 2 minutes after creation, the model frequently defaults to a low or 0.0 score.
• Aggregated Network Egress (Corporate VPNs, Universities, Mobile CGNAT): When multiple legitimate users share the same public egress IP, sudden bursts in traffic from that IP can resemble automated bot activity before the model learns the network density.
• Browser / Environment Signals: Privacy extensions, ad blockers, or new browser versions can cause the model to flag traffic with reason codes like UNEXPECTED_ENVIRONMENT.
• Generic or Missing Action Names: If all workflows use the same generic action name (such as 'homepage') or if your backend fails to validate that the action matches the expected transaction, the scoring model cannot differentiate simple page views from high-risk transaction attempts.

2. Does the model require a longer period to stabilize or learn?

Yes. While 2 to 3 days is the absolute minimum, Google's adaptive risk model typically requires at least 1 to 2 weeks of continuous traffic across weekdays, weekends, and peak hours to establish an accurate behavioral baseline.

3 & 4. How to manage score fluctuations and determine an appropriate fraud threshold?

Best Practice: Never hard-block users based on risk score alone. Use a 3-tier Step-Up Verification strategy:

• High Score (0.7 and above): Allow the transaction seamlessly.
• Medium Score (0.3 to 0.6): Trigger a Step-Up Challenge (e.g., SMS OTP, Email OTP, or a reCAPTCHA Enterprise Checkbox challenge).
• Low Score (below 0.3): Combine the score with your internal fraud indicators (e.g., new device ID, high transaction amount, rapid attempt velocity). If both indicate risk, route the transaction to a manual review queue or require multi-factor authentication.

5. What logs, assessments, and signals should you review to troubleshoot?

If you are using reCAPTCHA Enterprise:

• Inspect Server-Side Assessment Responses: Check tokenProperties.valid, ensure tokenProperties.action matches your expected action, and examine riskAnalysis.reasons (such as AUTOMATION, UNEXPECTED_ENVIRONMENT, TOO_MUCH_TRAFFIC, or LOW_CONFIDENCE_SCORE).
• Google Cloud Console Metrics: Go to Security > reCAPTCHA Enterprise in your Google Cloud Console. Filter score distributions by Action, Browser, and Operating System to identify whether low scores are concentrated in a specific environment.
• Use the Annotation API (AnnotateAssessment): Whenever your backend confirms whether a transaction was genuine (e.g., successful 2FA/payment clearance) or fraudulent (chargeback/fraud confirmed), send an annotation back to Google with LEGITIMATE or FRAUDULENT. This directly trains and tunes the machine learning model for your application.

Recommended Next Steps:

1. Ensure grecaptcha.execute runs with distinct, action-specific names (e.g., action: 'checkout_payment').
2. Ensure tokens are validated on your backend within 120 seconds.
3. Switch from binary blocking to step-up verification (OTP / secondary challenge) for lower scores.
4. Integrate the Annotation API to calibrate Google's scoring engine with your verified ground truth.

Hope this helps you stabilize your fraud prevention pipeline!