OBS Encoding Overloaded even on the fastest preset. Please help diagnose! (CPU: 1950x - GPU: 1080ti)

Used to stream on Twitch back in 2022 with no issues. Took a 3 year hiatus. Came back to stream and get "Encoding Overloaded" even on the fastest preset. I know my hardware is older, but I have a separate gaming pc. All my Threadripper is doing is encoding. I think it should be able to handle it, but I could be wrong.
66 Replies
імі
імі6d ago
what are your encoding settings?
Jedi Ryan
Jedi RyanOP6d ago
I've tried every combination at this point, which combination do you want to see? they all still give me an encoding overloaded error
імі
імі6d ago
just a screenshot of your OBS encoder settings
Jedi Ryan
Jedi RyanOP6d ago
I have it set as low as it can go and I am still seeing the error, which makes me think it's not the settings.
No description
імі
імі6d ago
does it work fine if you use your CPU? I have no issues encoding on a 1920x with fast preset
Jedi Ryan
Jedi RyanOP6d ago
I'm not encoding using the cpu, that is why I have a gpu. This is a dedicated encoidng pc, I'm not gaming on it.
імі
імі6d ago
yeah, but does it work? or does the error appear there too?
Technically Alex
Are you using a capture card or NDI to go from the gaming pc to the capture pc?
Jedi Ryan
Jedi RyanOP6d ago
capture card 4K Mk 2
імі
імі6d ago
what's your framerate?
Jedi Ryan
Jedi RyanOP6d ago
in OBS?
імі
імі6d ago
yeah
Jedi Ryan
Jedi RyanOP6d ago
60
Technically Alex
And OBS is updated?
Jedi Ryan
Jedi RyanOP6d ago
of course
Technically Alex
Can you post a log file from OBS Something is fishy
Technically Alex
Analyzer | OBS
OBS (Open Broadcaster Software) is free and open source software for video recording and live streaming. Stream to Twitch, YouTube and many other providers or record your own videos with high quality H264 / AAC encoding.
Technically Alex
I would disable HAGS, check the game capture source thing, and double check if hardware decoding/acceleration is enabled for your capture cards Based on the log it looks like that’s disabled Also, what’s your base canvas resolution? If it’s like 4K or something and you’re downscaling with the encoder that could potentially cause issues
Addie // EposVox
I'm not seeing anything immediately obvious, other than perhaps 1440p HEVC is just too much for it (though I was recording 4k60 HEVC videos with it when I had one) could be some weird pcie bandwidth issue or something But hard to tell, as streamlabs or whatever is complaining about timestamps is flooding the log so the log didn't even finish
7:51:42.218: preset: p7
17:51:42.218: tuning: hq
7:51:42.218: preset: p7
17:51:42.218: tuning: hq
well for one change that to p1 P7 should not be used, especially on such aging hardware if i am looking at this log in a vacuum, that would be the immediate difference-making change
Jedi Ryan
Jedi RyanOP6d ago
I have it set to P1, idk why the log file isn't updated
Addie // EposVox
there is no recording in this log with P1
Jedi Ryan
Jedi RyanOP6d ago
I'm trying to stream, not record. I obviously want to record, but I can't even just stream to Twitch like I used to.
Addie // EposVox
your stream didn't really drop any frames the frame drops were on the 1440p60 HEVC 60mbps P7 recording via Replay Buffer which, as I've stated, is too much for your GPU to handle alongside the stream it would appear
No description
Addie // EposVox
you might have it set to auto-start with your stream
Jedi Ryan
Jedi RyanOP6d ago
I do
Jedi Ryan
Jedi RyanOP6d ago
Are you telling me that I can't use replay buffer?
Addie // EposVox
I'm telling you you can't use those settings for Replay Buffer, alongside your stream (or possibly at all)
Jedi Ryan
Jedi RyanOP6d ago
can you suggest the settings that I should be using for my hardware?
Addie // EposVox
knock it to P1 & try again, set multipass to quarter-res & try again if all else fails consider replay buffering at 1080p P7 is incredibly intense to run, especially at 1440p on such an old GPU, and provides zero advantages for recording
Jedi Ryan
Jedi RyanOP6d ago
okay, so if I'm understanding you correctly. My recording settings were interfering with my stream because of replay buffer?
Addie // EposVox
yes the replay buffer is constantly recording alongside your stream they're both encoding the replay buffer encoding is too intense and lagging the nvenc encoder - which naturally affects other encode jobs happening at the same time
Jedi Ryan
Jedi RyanOP6d ago
okay, thank you for that clarity. that is the kind of answer I was looking for
Addie // EposVox
NP
Jedi Ryan
Jedi RyanOP6d ago
I never used to use replay buffer when I streamed back in 2022, so that is what has been causing the issues the whole time. Genuinely, thank you @Addie // EposVox
Addie // EposVox
Yeah haha you had a working setup and then threw the like worst case scenario for it on top of it hehe Despite how it may seem, the replay buffer is recording the whole time you're streaming, not just when it saves
Jedi Ryan
Jedi RyanOP6d ago
I honestly thought that it was just using RAM to save it since you have to allocate maximum memory is that RAM or VRAM that the replay buffer uses?
Addie // EposVox
ah no, it writes the recording to RAM (since it's faster than conventional storage) but it still has to use encoding hardware to do the encoding itself normal DRAM (system memory) not VRAM
Technically Alex
My replay buffer is I’m just always recording lol
Jedi Ryan
Jedi RyanOP6d ago
that's what I used to do, but then I realzied I was just scrubbing the recordings looking for clips, when the whole time I could of just used replay buffer to save the clips. kinda shot myself in the foot though lol
Addie // EposVox
yeah i have so little patience to find clips it's miserable to my adhd brain even when I'm saving shadowplay clips I struggle to sort
Technically Alex
This is the benefit of not having ADHD and also recording gameplay being my job lol I find the clips later bro I just need everything recorded
Addie // EposVox
i just need the budget to hire you and go "everything I reference is in one of these clips somewhere, assemble them for me" don't even gotta do the creative work, just putting the clips to the VO
Technically Alex
The problem is I’m already doing that lol I can’t do it for more than one person at a time bro I’d lose it
Jedi Ryan
Jedi RyanOP6d ago
I'm about to start testing again now that I know about the replay buffer being a factor. I thought of question before I get started. If I'm using NVENC H.264 for streaming and NVENC HEVC for recording via Replay Buffer, does it utilize the same part of my GPU or does it split the load?
імі
імі6d ago
afaik the 1080ti has two encoder engines how the load is shared between them depends on the software I guess
Jedi Ryan
Jedi RyanOP6d ago
I'm wondering if I'd be better off using cpu for streaming and use gpu for recording (Replay Buffer) or vice versa. I intend to multistream to YouTube in addition to Twitch. If I'm multistreaming to Twitch at 1080p60 and YouTube at 1440p60, plus using Replay Buffer to record at 1440p60, is it better to use gpu for streaming and cpu for recording (Replay Buffer) or vice versa. Would greatly appreciate your advice @//addie According to Google both NVENC H.264 and NVENC HEVC utilize the same encoder chip on the gpu, so I think my best bet is to use cpu for one and gpu for the other. I guess only testing will tell which is better. I just get overwhelmed by how many variables there are. Like, do I lower the preset or multipass mode first? Also, what about Look Ahead and Adaptive Quantization, do I uncheck them before or after chaning the preset/multipass mode??
Technically Alex
Yeah for max quality and best load distribution on a capture PC you can split the encoder load between the cpu and GPU For recording the lowest preset is always recommended Look ahead and AQ both take additional GPU resources for minimal visual gains
Jedi Ryan
Jedi RyanOP6d ago
as in lowest quality? that doesn't make sense to me.
Technically Alex
The presets doesn’t do anything useful when the recording is getting the proper bitrate The presets are tuning intended for bitrate constrained use cases like streaming on Twitch And even then…. Eh You’re basically just telling the encoder to spend more time working harder on each frame for a marginal gain in quality While giving up significantly more headroom for the hardware
Jedi Ryan
Jedi RyanOP6d ago
Interesting. I'll keep that in mind. Right now streaming on Twitch and the Replay Buffer going I'm at 70% GPU utilization. Trying to find a way to add YouTube without it overloading.
Technically Alex
You should put the 4070ti in for that lol
Jedi Ryan
Jedi RyanOP6d ago
Won't it be bottlenecked by my 1950x? I've been out of the tech space for a few years so my knowledge isn't up to date anymore.
імі
імі6d ago
the 1950x has plenty enough bandwidth to handle shoving data to the GPU for encoding you don't plan to run games on this machine, so no it won't be bottlenecked
Technically Alex
GPU encoding literally avoids using the CPU as much as possible That’s the whole point of it
Jedi Ryan
Jedi RyanOP6d ago
the only game I run on it is Slippi (Dolphin Emulator) For x264, should I be using main or high profile?
Addie // EposVox
Either
Jedi Ryan
Jedi RyanOP6d ago
what's the difference?
Addie // EposVox
Nothing relevant/meaningful
Jedi Ryan
Jedi RyanOP6d ago
I guess I'm at the limit of my hardware. I set OBS to encode Twitch with x264 and use NVENC for everything else. I'm able to stream to Twitch, YouTube & YouTube vertical with replay buffer on, but if I try to add TikTok, I get encoding overloaded.
MEE6
MEE66d ago
GG @Jedi Ryan, you just advanced to level 5 !
Addie // EposVox
Could have YouTube vertical and TikTok use the same encoder and save that performance But yes you’re asking about of your aged hardware
Jedi Ryan
Jedi RyanOP6d ago
I think I tried that. You mean setting both YouTube Vertical and TikTok to NVENC HEVC?
Addie // EposVox
There should be an option in whatever you’re using to just re-use the same encoder session
Jedi Ryan
Jedi RyanOP6d ago
I'm using Aitum Multistream. There is an option called "Veritcal Encoder". Maybe if I set TikTok to that it will re-use the encoder settings for YouTube Vertical?

Did you find this page helpful?