C
C#2y ago
kunio_kun

ASP NET Core 6 Launch without Debugging doesn't block Background Service but Debug Does

Hi, I'm trying to have a ASP.NET Core BackgroundService run really in background, i mean really don't interrupt with request/response part and I've been trying many stuff
private Thread? _workerThread;

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
Logger.LogInformation("Monitoring Event Triggering Start");
_workerThread = new Thread(() =>
{
while (!stoppingToken.IsCancellationRequested)
{
if (EventQueue.Count == 0) continue;
var monitoringEvent = EventQueue.Dequeue();
using (var scope = ServiceProvider.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
db.MonitoringEvents.Add(monitoringEvent);
db.SaveChanges();
}

// TODO Trigger events, notifications
Logger.LogInformation($"{monitoringEvent.DateTime} Rule {monitoringEvent.Message} triggered");
}
});
_workerThread.IsBackground = true;
_workerThread.Start();
}
private Thread? _workerThread;

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
Logger.LogInformation("Monitoring Event Triggering Start");
_workerThread = new Thread(() =>
{
while (!stoppingToken.IsCancellationRequested)
{
if (EventQueue.Count == 0) continue;
var monitoringEvent = EventQueue.Dequeue();
using (var scope = ServiceProvider.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
db.MonitoringEvents.Add(monitoringEvent);
db.SaveChanges();
}

// TODO Trigger events, notifications
Logger.LogInformation($"{monitoringEvent.DateTime} Rule {monitoringEvent.Message} triggered");
}
});
_workerThread.IsBackground = true;
_workerThread.Start();
}
That is the code i currently have, and one weird thing is if I debug the code, the service blocks the program execution, while if I use run, it doesn't block and continue as expected. Is there any mistakes there? Also, is there other ways to do this? I tried Task.Run(), Task.Factory.StartNew() and they all block. I'm on Jetbrains Rider on Ubuntu 20.04 if that matters for the threading part. Thanks in advance!
91 Replies
Kouhai
Kouhai2y ago
There's no way to achieve that, all threads are halted when reaching a breakpoint
kunio_kun
kunio_kun2y ago
ah it's not that i put any breakpoints there
kunio_kun
kunio_kun2y ago
i meant this buttons
kunio_kun
kunio_kun2y ago
If i started it with Debug, it never made it to the next service but it works correctly with Run
kunio_kun
kunio_kun2y ago
I have been investigating for a few hours and with Jetbrains' IDE i found that it doesn't get to the TryExecuteBackgroundServiceAsync() and the foreach loop stucks
Kouhai
Kouhai2y ago
That's weird, do you have specific code that runs for development vs production builds? Does the foreach loop even execute or is _hostedServices empty?
kunio_kun
kunio_kun2y ago
it is not empty I have 2 background services and with the debugger breakpoint it didn't evaluate to true for the first background service (this one) also i dont have any specific code that runs on different configuration
Kouhai
Kouhai2y ago
does hostedService.StartAsync await infinitely? (or it's not even reached)
kunio_kun
kunio_kun2y ago
at least not in my own code (non library), and not in this service does it refer to this backgroundservice? it is reached it does have infinite loop I have updated the code to this
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (LongRunningTask is not null) throw new InvalidOperationException("Already running");

Logger.LogInformation("Monitoring Event Triggering Start");
LongRunningTask = Task.Run(async () =>
{
while (!stoppingToken.IsCancellationRequested)
{
if (EventQueue.Count == 0) continue;
var monitoringEvent = EventQueue.Dequeue();
using (var scope = ServiceProvider.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
await db.MonitoringEvents.AddAsync(monitoringEvent, stoppingToken);
await db.SaveChangesAsync(stoppingToken);
}

// TODO Trigger events, notifications
Logger.LogInformation($"{monitoringEvent.DateTime} Rule {monitoringEvent.Message} triggered");
}
},
stoppingToken
);
Logger.LogInformation("Task is running");
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (LongRunningTask is not null) throw new InvalidOperationException("Already running");

Logger.LogInformation("Monitoring Event Triggering Start");
LongRunningTask = Task.Run(async () =>
{
while (!stoppingToken.IsCancellationRequested)
{
if (EventQueue.Count == 0) continue;
var monitoringEvent = EventQueue.Dequeue();
using (var scope = ServiceProvider.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
await db.MonitoringEvents.AddAsync(monitoringEvent, stoppingToken);
await db.SaveChangesAsync(stoppingToken);
}

// TODO Trigger events, notifications
Logger.LogInformation($"{monitoringEvent.DateTime} Rule {monitoringEvent.Message} triggered");
}
},
stoppingToken
);
Logger.LogInformation("Task is running");
}
ooh
kunio_kun
kunio_kun2y ago
kunio_kun
kunio_kun2y ago
now wonder if it blocks somewhere else but according to the debugger it doesnt even get to the second backgroundservice i have
Kouhai
Kouhai2y ago
Based on these logs, I assume both services seem to be started
kunio_kun
kunio_kun2y ago
i will try putting breakpoint again
kunio_kun
kunio_kun2y ago
kunio_kun
kunio_kun2y ago
these are the hostedServices
kunio_kun
kunio_kun2y ago
kunio_kun
kunio_kun2y ago
kunio_kun
kunio_kun2y ago
kunio_kun
kunio_kun2y ago
the next step after the last picture is nowhere
kunio_kun
kunio_kun2y ago
and the log is also stuck here
kunio_kun
kunio_kun2y ago
that was Host.cs, class Host in Microsoft.Extensions.Hosting.Internal but i mean even having breakpoints vs no breakpoints on debug has different result?
Kouhai
Kouhai2y ago
Hmm so it's stuck after calling StartAsync on the MonitoringEventTriggeringService, I'm not sure if Host.cs has any conditions for production vs development but I doubt that
kunio_kun
kunio_kun2y ago
it does get to
if (hostedService is BackgroundService backgroundService)
if (hostedService is BackgroundService backgroundService)
and then next step is nowhere
Kouhai
Kouhai2y ago
Oh
kunio_kun
kunio_kun2y ago
okay just tried it again this time the next step was the closing curly brackets of the foreach loop, as if the hostedService is not BackgroundService got to the next enumeration (is it?) in the in _hostedService and then next step is nowhere
Kouhai
Kouhai2y ago
The condition hostedService is BackgroundService backgroundServic fails? Hmm even if it fails it should continue looping
kunio_kun
kunio_kun2y ago
just tried again it didnt continue after the hostedService is BackgroundService part i mean even if it fails, it may be throwing something right?
Kouhai
Kouhai2y ago
But what would throw? is T just checks if the instance is of that type, if so it would run the code inside the braces
kunio_kun
kunio_kun2y ago
while it does inherit from BackgroundService
Kouhai
Kouhai2y ago
idk if it would help, but what version of ASP.NET Core are you using
kunio_kun
kunio_kun2y ago
ASP.NET Core MVC 6
dotnet --version
6.0.402
dotnet --version
6.0.402
Linux 5.15.0-52-generic #58~20.04.1-Ubuntu SMP Thu Oct 13 13:09:46 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Linux 5.15.0-52-generic #58~20.04.1-Ubuntu SMP Thu Oct 13 13:09:46 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
maybe i should also try it on Windows and see if it happens
Kouhai
Kouhai2y ago
This SS is for the release build, right thisisalright
kunio_kun
kunio_kun2y ago
nope
Kouhai
Kouhai2y ago
Okay..in that pic it ModBus is run normally
kunio_kun
kunio_kun2y ago
if i didnt put breakpoint at all, even with Debug, it reaches that part it does start, but it doesnt run normally like it reaches some point and not continue but if i put some breakpoints and it hit, it doesnt get to even initiate the next service and sometimes it doesnt get to that part of log too (ends on Task is running) on debug
Kouhai
Kouhai2y ago
Can you do something real quick, wrap all your code that uses Task.Run with try catch, and in catch log any message
kunio_kun
kunio_kun2y ago
mmmm wrap in the Task.Run() delegate? so like
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (LongRunningTask is not null) throw new InvalidOperationException("Already running");

Logger.LogInformation("Monitoring Event Triggering Start");
LongRunningTask = Task.Run(async () =>
{
// start wrapping here?
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (LongRunningTask is not null) throw new InvalidOperationException("Already running");

Logger.LogInformation("Monitoring Event Triggering Start");
LongRunningTask = Task.Run(async () =>
{
// start wrapping here?
Kouhai
Kouhai2y ago
Yeah
kunio_kun
kunio_kun2y ago
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (LongRunningTask is not null) throw new InvalidOperationException("Already running");

Logger.LogInformation("Monitoring Event Triggering Start");
LongRunningTask = Task.Run(async () =>
{
try
{
while (!stoppingToken.IsCancellationRequested)
{
if (EventQueue.Count == 0) continue;
var monitoringEvent = EventQueue.Dequeue();
using (var scope = ServiceProvider.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
await db.MonitoringEvents.AddAsync(monitoringEvent, stoppingToken);
await db.SaveChangesAsync(stoppingToken);
}

// TODO Trigger events, notifications
Logger.LogInformation($"{monitoringEvent.DateTime} Rule {monitoringEvent.Message} triggered");
}
}
catch (Exception ex)
{
Logger.LogError(ex, "Something went wrong");
}
},
stoppingToken
);
Logger.LogInformation("Task is running");
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (LongRunningTask is not null) throw new InvalidOperationException("Already running");

Logger.LogInformation("Monitoring Event Triggering Start");
LongRunningTask = Task.Run(async () =>
{
try
{
while (!stoppingToken.IsCancellationRequested)
{
if (EventQueue.Count == 0) continue;
var monitoringEvent = EventQueue.Dequeue();
using (var scope = ServiceProvider.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
await db.MonitoringEvents.AddAsync(monitoringEvent, stoppingToken);
await db.SaveChangesAsync(stoppingToken);
}

// TODO Trigger events, notifications
Logger.LogInformation($"{monitoringEvent.DateTime} Rule {monitoringEvent.Message} triggered");
}
}
catch (Exception ex)
{
Logger.LogError(ex, "Something went wrong");
}
},
stoppingToken
);
Logger.LogInformation("Task is running");
}
Kouhai
Kouhai2y ago
Do you have any other service/s that calls Task.Run?
kunio_kun
kunio_kun2y ago
i dont I have also commented out other services
kunio_kun
kunio_kun2y ago
kunio_kun
kunio_kun2y ago
still it didnt make it to the web service
using RemoteMonitor.DatabaseContexts;
using RemoteMonitor.Services;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllersWithViews();
// .AddRazorRuntimeCompilation();
builder.Services.AddDbContext<AppDbContext>();
// builder.Services.AddHostedService<DatabaseMigratorService>();
// builder.Services.AddSignalR().AddMessagePackProtocol();
builder.Services.AddSingleton<MonitoringEventTriggeringService>();
builder.Services.AddHostedService(provider => provider.GetRequiredService<MonitoringEventTriggeringService>());
// builder.Services.AddSingleton<ModbusDeviceWorkerService>();
// builder.Services.AddHostedService(provider => provider.GetRequiredService<ModbusDeviceWorkerService>());

var app = builder.Build();

if (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler("/Home/Error");
app.UseHsts();
}

// app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthorization();

app.MapControllerRoute(
name: "default",
pattern: "{controller=Home}/{action=Index}/{id?}"
);

app.Run();
using RemoteMonitor.DatabaseContexts;
using RemoteMonitor.Services;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllersWithViews();
// .AddRazorRuntimeCompilation();
builder.Services.AddDbContext<AppDbContext>();
// builder.Services.AddHostedService<DatabaseMigratorService>();
// builder.Services.AddSignalR().AddMessagePackProtocol();
builder.Services.AddSingleton<MonitoringEventTriggeringService>();
builder.Services.AddHostedService(provider => provider.GetRequiredService<MonitoringEventTriggeringService>());
// builder.Services.AddSingleton<ModbusDeviceWorkerService>();
// builder.Services.AddHostedService(provider => provider.GetRequiredService<ModbusDeviceWorkerService>());

var app = builder.Build();

if (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler("/Home/Error");
app.UseHsts();
}

// app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthorization();

app.MapControllerRoute(
name: "default",
pattern: "{controller=Home}/{action=Index}/{id?}"
);

app.Run();
Hitting the pause button on debug doesnt even pause it Ctrl + C doesnt shut it down
Tvde1
Tvde12y ago
You can replace
builder.Services.AddHostedService(provider => provider.GetRequiredService<MonitoringEventTriggeringService>());
builder.Services.AddHostedService(provider => provider.GetRequiredService<MonitoringEventTriggeringService>());
with
builder.Services.AddHostedService<MonitoringEventTriggeringService>();
builder.Services.AddHostedService<MonitoringEventTriggeringService>();
right?
Kouhai
Kouhai2y ago
Yeah, HostedServices are singletons by default
kunio_kun
kunio_kun2y ago
but I wasnt able to resolve it on the other service without registering it as singleton also I now have this instead to try it out:
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllersWithViews();
// .AddRazorRuntimeCompilation();
builder.Services.AddDbContext<AppDbContext>();
// builder.Services.AddHostedService<DatabaseMigratorService>();
// builder.Services.AddSignalR().AddMessagePackProtocol();
builder.Services.AddHostedService<MonitoringEventTriggeringService>();
// builder.Services.AddSingleton<MonitoringEventTriggeringService>();
// builder.Services.AddHostedService(provider => provider.GetRequiredService<MonitoringEventTriggeringService>());
// builder.Services.AddSingleton<ModbusDeviceWorkerService>();
// builder.Services.AddHostedService(provider => provider.GetRequiredService<ModbusDeviceWorkerService>());
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllersWithViews();
// .AddRazorRuntimeCompilation();
builder.Services.AddDbContext<AppDbContext>();
// builder.Services.AddHostedService<DatabaseMigratorService>();
// builder.Services.AddSignalR().AddMessagePackProtocol();
builder.Services.AddHostedService<MonitoringEventTriggeringService>();
// builder.Services.AddSingleton<MonitoringEventTriggeringService>();
// builder.Services.AddHostedService(provider => provider.GetRequiredService<MonitoringEventTriggeringService>());
// builder.Services.AddSingleton<ModbusDeviceWorkerService>();
// builder.Services.AddHostedService(provider => provider.GetRequiredService<ModbusDeviceWorkerService>());
still stuck there
Tvde1
Tvde12y ago
that's a lot of commented out code :)
kunio_kun
kunio_kun2y ago
yeah
Tvde1
Tvde12y ago
but what's the problem? You don't see console output from the task?
kunio_kun
kunio_kun2y ago
Nope
Tvde1
Tvde12y ago
you might want to put a timer in the while, not to check EventQueue.Count every nano second
kunio_kun
kunio_kun2y ago
the app seems seems to be stuck because this service yeah doing this works, like putting a Task.Delay() and await it
Tvde1
Tvde12y ago
that works but is not as robust, the PeriodicTimer is really good for these use cases
kunio_kun
kunio_kun2y ago
but still it is weird that if I start the app without debugging it, it works
kunio_kun
kunio_kun2y ago
run without debugging
Tvde1
Tvde12y ago
but you shouldn't have to make a thread or task in your ExecuteAsync
kunio_kun
kunio_kun2y ago
meanwhile run with debugging
kunio_kun
kunio_kun2y ago
mmmm
Tvde1
Tvde12y ago
hmm are you hitting a break point during debugging?
kunio_kun
kunio_kun2y ago
i dont but i mean if the host is implemented this way, and the service is background service, then it shouldn't be blocking right?
Tvde1
Tvde12y ago
yeah I think it's not blocking it should create a thread for the background service
kunio_kun
kunio_kun2y ago
even if i create a thread, with IsBackground set to true it still blocks
Tvde1
Tvde12y ago
hmmm
kunio_kun
kunio_kun2y ago
but not if i start without debugging still no luck, i tried with break when any exception is thrown also did another try with mute breakpoints didnt see any clue
Kouhai
Kouhai2y ago
hmmm, can you try adding a log message at the top of Task.Run delegate?
kunio_kun
kunio_kun2y ago
sure
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (LongRunningTask is not null) throw new InvalidOperationException("Already running");

Logger.LogInformation("Monitoring Event Triggering Start");
LongRunningTask = Task.Run(async () =>
{
Logger.LogInformation("Task Start");
try
{
while (!stoppingToken.IsCancellationRequested)
{
if (EventQueue.Count == 0)
{
continue;
}

var monitoringEvent = EventQueue.Dequeue();
using (var scope = ServiceProvider.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
await db.MonitoringEvents.AddAsync(monitoringEvent, stoppingToken);
await db.SaveChangesAsync(stoppingToken);
}

// TODO Trigger events, notifications
Logger.LogInformation($"{monitoringEvent.DateTime} Rule {monitoringEvent.Message} triggered");
}
}
catch (Exception ex)
{
Logger.LogError(ex, "Something went wrong");
}
},
stoppingToken
);
Logger.LogInformation("Task is running");
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (LongRunningTask is not null) throw new InvalidOperationException("Already running");

Logger.LogInformation("Monitoring Event Triggering Start");
LongRunningTask = Task.Run(async () =>
{
Logger.LogInformation("Task Start");
try
{
while (!stoppingToken.IsCancellationRequested)
{
if (EventQueue.Count == 0)
{
continue;
}

var monitoringEvent = EventQueue.Dequeue();
using (var scope = ServiceProvider.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
await db.MonitoringEvents.AddAsync(monitoringEvent, stoppingToken);
await db.SaveChangesAsync(stoppingToken);
}

// TODO Trigger events, notifications
Logger.LogInformation($"{monitoringEvent.DateTime} Rule {monitoringEvent.Message} triggered");
}
}
catch (Exception ex)
{
Logger.LogError(ex, "Something went wrong");
}
},
stoppingToken
);
Logger.LogInformation("Task is running");
}
kunio_kun
kunio_kun2y ago
kunio_kun
kunio_kun2y ago
also tried with
try
{
while (!stoppingToken.IsCancellationRequested)
{
Logger.LogInformation("Task alive");
try
{
while (!stoppingToken.IsCancellationRequested)
{
Logger.LogInformation("Task alive");
and it's spamming Task alive
Kouhai
Kouhai2y ago
That's really weird... 😅 I'm honestly lost at what could cause that....
kunio_kun
kunio_kun2y ago
using logger to log Task alive blocks but weirdly i have that Console.Write(".") that somehow caused it not to block
kunio_kun
kunio_kun2y ago
kunio_kun
kunio_kun2y ago
But thanks a lot for at least having a look
Tvde1
Tvde12y ago
where did you write the .? in what line of code? but Monitoring Event Triggering Start or Already running is not logged?
kunio_kun
kunio_kun2y ago
In the while loop It is, everything is logged, just covered with the dot spam from that service
Tvde1
Tvde12y ago
try adding a await Task.Yield(); in the while loop, I'm curious
kunio_kun
kunio_kun2y ago
Will try when i'm back to linux Now i'm trying in Windows and i can debug But my Rider version is a few version older Updating it .NET 6 version is also a few version older on my windows machine anyway i also tried this thing before on linux: I started it without debugging, then attach the debugger on the currently running ASP.NET Core 6 app with this same, then attach the debugger And the service stops updated Rider and SDK version on Windows and it still can debug Seems to be linux specific WOW
kunio_kun
kunio_kun2y ago
kunio_kun
kunio_kun2y ago
tried this
LongRunningTask = Task.Run(async () =>
{
Task.Yield();
Logger.LogInformation("Task Start");
try
{
while (!stoppingToken.IsCancellationRequested)
{
Task.Yield();
if (EventQueue.Count == 0)
{
continue;
}
LongRunningTask = Task.Run(async () =>
{
Task.Yield();
Logger.LogInformation("Task Start");
try
{
while (!stoppingToken.IsCancellationRequested)
{
Task.Yield();
if (EventQueue.Count == 0)
{
continue;
}
Tvde1
Tvde12y ago
await the Task.Yield
kunio_kun
kunio_kun2y ago
oh sorry
LongRunningTask = Task.Run(async () =>
{
await Task.Yield();
Logger.LogInformation("Task Start");
try
LongRunningTask = Task.Run(async () =>
{
await Task.Yield();
Logger.LogInformation("Task Start");
try
seems to be the same but having it in the while loop helps
LongRunningTask = Task.Run(async () =>
{
await Task.Yield();
Logger.LogInformation("Task Start");
try
{
while (!stoppingToken.IsCancellationRequested)
{
await Task.Yield();
LongRunningTask = Task.Run(async () =>
{
await Task.Yield();
Logger.LogInformation("Task Start");
try
{
while (!stoppingToken.IsCancellationRequested)
{
await Task.Yield();
I'm using await Task.Yield() for now
kunio_kun
kunio_kun2y ago
I also opened an issue at Github https://github.com/dotnet/aspnetcore/issues/44706
GitHub
Launch Without Debugging Doesn't Block BackgroundService and Whole ...
Is there an existing issue for this? I have searched the existing issues Describe the bug I have a BackgroundService that looks like this using RemoteMonitor.Classes; using RemoteMonitor.DatabaseCo...
Kouhai
Kouhai2y ago
It seems like it's a runtime problem Even a very simple code like this
for (int i = 0; i < 3; i++)
{
Task.Run(() =>
{
Console.WriteLine("Task started");
while (true) { }
});
await Task.Delay(500);
}
for (int i = 0; i < 3; i++)
{
Task.Run(() =>
{
Console.WriteLine("Task started");
while (true) { }
});
await Task.Delay(500);
}
Halts after the first Task is started I tested it on WSL running Debian distro
kunio_kun
kunio_kun2y ago
I see even without ASP.NET Core 6?
Kouhai
Kouhai2y ago
Yup, just a very simple console app
kunio_kun
kunio_kun2y ago
i see if you don't mind i'll include these as well into the issue
Kouhai
Kouhai2y ago
Yeah sure, go ahead
kunio_kun
kunio_kun2y ago
thanks
Kouhai
Kouhai2y ago
It was also "fixed" by using await Task.Delay or await Task.Yield, so it seems like infinite loops makes something break on Linux
kunio_kun
kunio_kun2y ago
well not so infinite on my previous code, at least checks for stoppingToken
Kouhai
Kouhai2y ago
True, but practically it was infinite, because stoppingToken doesn't cancel before the whole program halts
kunio_kun
kunio_kun2y ago
ah yeah