Dear @AThebrand,

I can see multiple approach here:
1) You can configure the case aggregation mecanism in order to stop alerts being aggregated into the same case.
2) If I'm not mistaken, you can configure a playbook in order to run only for the first alert in a case.
3) In your playbook, you can close the alert (and so case will be closed if it's the latest alert in the case) after creating the Jira ticket so that any new alert will create a new case.

That being said, your playbook use-case is not very clear to me so maybe my answers are not relevant.
In our deployment we are more using a one playbook per type of alert and we are attaching other playbooks from the first one if needed.
In this situation, we are very glad that our playbooks are triggered per alert and not per case 😉

Louis

We had this problem at one of my previous companies.  The connector was SentinelOne and the ticketing system was ServiceNow.  We definitely wanted to group alerts into cases but only have one ServiceNow ticket per case.  We worked with Siemplify professional services to create an action that would determine if an alert was the first one in a case.  A conditional then would lead to a ServiceNow action if "true" was returned.  

This work may be now contained with in the Tools Power Up in the SOAR Marketplace.  See if "Find First Alert" in Tools does what I'm describing.  Either way, it's possible though.