Hi Google Cloud Community,
We recently enabled reCAPTCHA v3 as a fraud-prevention mechanism and monitored the risk scores for approximately two weeks to understand its assessment behavior.
During the first two days, we did not consider the scores for evaluation, as we followed the recommendation in the documentation to allow sufficient time for the risk analysis/assessment to establish.
For the following five days, the results were as expected:
- Fraudulent transactions: The reCAPTCHA score was consistently 0.4 or below.
- Genuine transactions: The score was consistently 0.5 or above.
- This separation between fraudulent and genuine transactions was useful for defining our fraud-detection threshold.
However, after approximately five days, we started seeing inconsistent results. Our genuine transactions are now also receiving very low scores, sometimes as low as 0.0, even though these transactions are legitimate.
This behavior is making it difficult for us to reliably distinguish between genuine and fraudulent transactions based on the reCAPTCHA v3 score.
Could someone please help us understand:
- What could cause genuine transactions to start receiving significantly lower scores after initially receiving expected scores?
- Does the reCAPTCHA v3 risk assessment model require a longer period of time to stabilize or learn traffic patterns?
- Since traffic patterns naturally vary due to seasonality and cannot be treated as a fixed baseline, how should we interpret score fluctuations in such dynamic conditions, and what approach is recommended for setting a stable threshold when traffic behavior is continuously changing?
- What is the recommended approach for determining an appropriate fraud threshold when genuine transactions can receive scores as low as 0.0?
- Are there any logs, assessments, or additional signals we should review to troubleshoot why the scores have changed?
We would appreciate any guidance on how we can investigate this behavior and ensure that reCAPTCHA v3 provides a reliable risk assessment for our fraud-prevention use case.
Thank you.
