Same old problem; even though I wish it didn’t exist, I’m glad to see I’m not the only one experiencing it.
I have recently insisted on using a click track at all rehearsals - it helps you to figure out who is rushing and/or dragging.
Most non-drummers are just not aware enough of the tempo.
Luckily, most of the peeps I play with have “seen the light” and are now practicing at home with a metronome. Yeah!