Cav's lost the Gold Jersey at the Tour of Britain after a wind hit end to the third stage.
Lars Boom and Michael Matthews took first and second for Rabobank in Stoke-on-Trent.
A late attack by Sky's Alex Dowsett was chased down by the leading pack in the face of strong winds.
The uphill sprint to the finish turned into a bit of a melee with Dutchman Boom taking the line honours, Matthews second, Geraint Thomas third and Stephen Cummings fourth. Cavendish was fifth.
Boom's win edges Cavendish out at the top of the sprint leaderboard but he still starts Wednesday's Welshpool to Caerphilly stage in the points jersey.