If I am using Workers Websocket without Durable Objects (for example using it for listening to queues or as a proxy for another websocket server), how would the pricing work? Is it charged based on per request (hence per websocket connection) and the cpu-time used to process the websocket messages?
Yes. But keep in mind this is CPU time, not wall time (not time as measured on a clock but time that is measured based on how much compute you are doing)
Unless you have an endless for loop running in your worker to eat CPU time, you might have just measured real (wall) time, which does not matter. Workers can run for a year I think if they do not consume CPU, see -> https://developers.cloudflare.com/workers/platform/limits/#duration
You get 30 seconds of cpu time, so that means that if your WebSocket isn't doing much it could theoretically last for many minutes, although it's not a 100% because sometimes websockets are killed by Cloudflare for internal reasons.