I am assuming this is online. If that is true then don’t assume someone was thinking of calling prior hands or even remembers them. So many online players multi-table and or just flat out multi task it’s a bad assumption that they are paying attention. If they had timing issues prior it only increases the likely hood they are multi-tabling. With that information just play ABC online at those stakes. It’s boring but you will profit and move up.
Now if this is like a live home game then this is fine to switch it up once in a while.