hostapd won't start via systemctl

I'm having a weird issue with hostapd. When I run it via systemctl start hostapd, it fails with this log:
Jan 03 19:39:48 fedora systemd[1]: Starting hostapd.service - Hostapd IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator...
Jan 03 19:39:48 fedora hostapd[1435]: Could not open configuration file '/etc/hostapd/hostapd.conf' for reading.
Jan 03 19:39:48 fedora hostapd[1435]: Failed to set up interface with /etc/hostapd/hostapd.conf
Jan 03 19:39:48 fedora hostapd[1435]: Failed to initialize interface
Jan 03 19:39:48 fedora systemd[1]: hostapd.service: Control process exited, code=exited, status=1/FAILURE
Jan 03 19:39:48 fedora systemd[1]: hostapd.service: Failed with result 'exit-code'.
Jan 03 19:39:48 fedora systemd[1]: Failed to start hostapd.service - Hostapd IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator.
Jan 03 19:39:48 fedora systemd[1]: Starting hostapd.service - Hostapd IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator...
Jan 03 19:39:48 fedora hostapd[1435]: Could not open configuration file '/etc/hostapd/hostapd.conf' for reading.
Jan 03 19:39:48 fedora hostapd[1435]: Failed to set up interface with /etc/hostapd/hostapd.conf
Jan 03 19:39:48 fedora hostapd[1435]: Failed to initialize interface
Jan 03 19:39:48 fedora systemd[1]: hostapd.service: Control process exited, code=exited, status=1/FAILURE
Jan 03 19:39:48 fedora systemd[1]: hostapd.service: Failed with result 'exit-code'.
Jan 03 19:39:48 fedora systemd[1]: Failed to start hostapd.service - Hostapd IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator.
The file definitely exists though, and if I run the command from hostapd.service manually, it works fine:
$ sudo /usr/sbin/hostapd /etc/hostapd/hostapd.conf -P /run/hostapd.pid
wlp9s0f4u2u1: interface state UNINITIALIZED->COUNTRY_UPDATE
wlp9s0f4u2u1: interface state COUNTRY_UPDATE->ENABLED
wlp9s0f4u2u1: AP-ENABLED
...
$ sudo /usr/sbin/hostapd /etc/hostapd/hostapd.conf -P /run/hostapd.pid
wlp9s0f4u2u1: interface state UNINITIALIZED->COUNTRY_UPDATE
wlp9s0f4u2u1: interface state COUNTRY_UPDATE->ENABLED
wlp9s0f4u2u1: AP-ENABLED
...
Changing the conf file perms to 644 didn't help, and searching for the error message didn't turn up anything about weird bugs. Is there something immutable filesystem-specific that could be causing this? hostapd was installed via rpm-ostree, if that makes a difference.
50 Replies
Noel
Noel•5mo ago
Are you running the sudo systemctl start hostapd.service or just systemctl start hostapd.service? also what does the systemd service file look like?
Evening Newbs
Evening Newbs•5mo ago
$ cat /usr/lib/systemd/system/hostapd.service
[Unit]
Description=Hostapd IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator
After=network.target

[Service]
Type=forking
PIDFile=/run/hostapd.pid
EnvironmentFile=/etc/sysconfig/hostapd
ExecStart=/usr/sbin/hostapd /etc/hostapd/hostapd.conf -P /run/hostapd.pid -B $OTHER_ARGS

[Install]
WantedBy=multi-user.target
$ cat /usr/lib/systemd/system/hostapd.service
[Unit]
Description=Hostapd IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator
After=network.target

[Service]
Type=forking
PIDFile=/run/hostapd.pid
EnvironmentFile=/etc/sysconfig/hostapd
ExecStart=/usr/sbin/hostapd /etc/hostapd/hostapd.conf -P /run/hostapd.pid -B $OTHER_ARGS

[Install]
WantedBy=multi-user.target
And it fails even if it starts at boot. Just retried sudo systemctl restart hostapd.service to be sure and it logged the same error.
Noel
Noel•5mo ago
that's really weird, nothing jumps out super obviously in the service file that would cause the issue. I wonder if it's an SELinux issue? could you try running sudo getenforce and see if SELinux is turned on? If it's enforcing, try doing sudo setenforce permissive and then try running sudo systemctl stop hosapd.service and then sudo systemctl start hosapd.service could be that the SELinux label isn't set correctly for whatever reason. if it was SELinux, I would think the behavior would persist even when you are trying to do it manually instead of the service file, but it would be nice to rule it out.
Evening Newbs
Evening Newbs•5mo ago
Whoa, that was it. I never would have guessed that. Thank you so much! Will that persist after a reboot?
Noel
Noel•5mo ago
no, we will probably want to do some more digging to figure out what we need to label the config file with to make it work. lemme try and find what commands to run to see the SELinux errors. Otherwise if you have cockpit installed, it has a nice interface to see that stuff! here it is try running sealert -a /var/log/audit/audit.log and paste here what it outputs. nice thing about this command is it will usually give you a command to run to fix it if it's simple.
Evening Newbs
Evening Newbs•5mo ago
I don't have a sealart command.
Noel
Noel•5mo ago
huh... I wonder if bazzite doesn't ship the setroubleshoot and setools packages. I have them on Bluefin-dx you may need to layer those packages in. I'll double check my bazzite machine, hold on.
Evening Newbs
Evening Newbs•5mo ago
Install it with rpm-ostree? Says they're available.
Noel
Noel•5mo ago
yeah, they do not appear to be built into the bazzite image. @KyleGospo @EyeCantCU we should definitly get those packages in for troubleshooting issues like this. this may be a quick PR we can do instead of having you layer them, but not totally sure on it. I think I found in the container file where to add the packages. I'll put together a quick PR. @Evening Newbs it should be in the next build of Bazzite. For now your options are: 1. Layering the package to continue troubleshooting 2. Set SELinux to permissive for now and continue troubleshooting once the new build ships.
Evening Newbs
Evening Newbs•5mo ago
If you layer something, can you revert it, or is the only option uninstalling and making a second layer? I've been wondering that for a while.
Noel
Noel•5mo ago
rpm-ostree uninstall should allow you to remove the layer. or the package rather.
Evening Newbs
Evening Newbs•5mo ago
Yeah, but is that 2 layers, or did you remove the layer you added? I'm mostly just curious.
1/4 Life
1/4 Life•5mo ago
There's only one layer, and that's the changes you've made Removing something simply remove it from your layer It's kind of like a git rebase if you've used git before
Evening Newbs
Evening Newbs•5mo ago
Ah, okay. I was picturing something more like a git commit for each rpm-ostree action.
1/4 Life
1/4 Life•5mo ago
It's technically like that, but removal is less a revert commit and more deleting it Unless you're removing an upstream included package
Evening Newbs
Evening Newbs•5mo ago
And rpm-ostree uninstall looking more like git revert in that it makes a commit to undo another one. Got it. Good to know.
Noel
Noel•5mo ago
I'm surprised base silverblue doesn't ship the SELinux tools. especially since it ships SELinux by default.
Evening Newbs
Evening Newbs•5mo ago
Oh, and to answer to original question, I'm happy to layer them and troubleshoot it now.
Noel
Noel•5mo ago
Go ahead and layer setroubleshoot and setools packages, then run sudo sealert -a /var/log/audit/audit.log
Evening Newbs
Evening Newbs•5mo ago
[Errno 13] Permission denied: '/var/log/audit/audit.log'
[Errno 13] Permission denied: '/var/log/audit/audit.log'
OH, missed sudo.
Noel
Noel•5mo ago
I missed it initially and then corrected myself 😄
Evening Newbs
Evening Newbs•5mo ago
These look like the relevant logs:
Noel
Noel•5mo ago
ok sick! so it looks like you just need to do a restorecon on the config file.
Evening Newbs
Evening Newbs•5mo ago
Ah, is this because I wrote the config elsewhere and copied it over?
Noel
Noel•5mo ago
That is exactly why!
Evening Newbs
Evening Newbs•5mo ago
Did I blow away some kind of hidden file property?
Noel
Noel•5mo ago
do me a favor and run ls -alZ /etc/hostapd that should show the current selinux label on the folder.
Evening Newbs
Evening Newbs•5mo ago
drwxr-xr-x. 1 root root system_u:object_r:etc_t:s0 24 Jan 4 19:51 .
drwxr-xr-x. 1 root root system_u:object_r:etc_t:s0 4906 Jan 4 19:51 ..
-rw-------. 1 root root unconfined_u:object_r:user_home_t:s0 128970 Jan 3 19:31 hostapd.conf
drwxr-xr-x. 1 root root system_u:object_r:etc_t:s0 24 Jan 4 19:51 .
drwxr-xr-x. 1 root root system_u:object_r:etc_t:s0 4906 Jan 4 19:51 ..
-rw-------. 1 root root unconfined_u:object_r:user_home_t:s0 128970 Jan 3 19:31 hostapd.conf
Noel
Noel•5mo ago
ok, so the folder has the right label, but you see how your hostapd.conf file has :user_home_t label instead of :etc_t?
Evening Newbs
Evening Newbs•5mo ago
Yeahhhhhh. Makes sense now.
Noel
Noel•5mo ago
sudo /sbin/restorecon -v /etc/hostapd/hostapd.conf should fix that label you can rerun ls -alZ /etc/hostapd to see if the change worked!
Evening Newbs
Evening Newbs•5mo ago
Yeah, it's etc_t now.
Noel
Noel•5mo ago
then you will want to do sudo setenforce enforcing to set selinux back to enforcing mode. and then try seeing if sudo systemctl restart hostapd will not give you the error anymore!
Evening Newbs
Evening Newbs•5mo ago
Yep, it works.
Noel
Noel•5mo ago
yay!
Evening Newbs
Evening Newbs•5mo ago
Thanks so much!
Noel
Noel•5mo ago
If it's not DNS, it's SELinux 😄
Evening Newbs
Evening Newbs•5mo ago
I really appreciate it. I feel like this is the same "welcome" everyone receives to SELinux.
Noel
Noel•5mo ago
No problem! Good rule of thumb is anytime there is a weird permissions issue that doesn't make sense, always try setting selinux to "permissive" and see if it fixes your issue. Glad I could help you figure it out!
Evening Newbs
Evening Newbs•5mo ago
Should I rpm-ostree uninstall those tools, or will it sort itself out when I update?
1/4 Life
1/4 Life•5mo ago
uninstall, then update it'll complain if you don't
Noel
Noel•5mo ago
Yeah, I was wondering the same thing! Good catch! Thanks @KyleGospo
Evening Newbs
Evening Newbs•5mo ago
Done. Thanks!
1/4 Life
1/4 Life•5mo ago
Thank you!
Evening Newbs
Evening Newbs•5mo ago
One unrelated question while we're here: will there be any ill effects if I disable NetworkManager and use systemd-networkd instead?
Noel
Noel•5mo ago
I certainly have not tried that. I'm assuming the effects would be similar to if you made this change in silverblue proper.
M2
M2•5mo ago
Did you copy or move the file over? If you copied it, it should of had the right type. If you every have issues with a systemd service. Open another terminal and run journalctl -xe. Start the troublesome service and it will be very obvious it's a SELinux issue. It will also tell you what might be the action to take (though that might require setroubleshoot). If it starts telling you to use audit2allow, pause and research. Restorecon has been fine. Honestly, probably should point people towards SELinux documentation since most people unless they came from fedora have zero experience with it.
Evening Newbs
Evening Newbs•5mo ago
I can't remember if I copied it or moved it. I do remember that I needed to adjust permissions once it was there, so probably moved.
M2
M2•5mo ago
Yeah don't move things. Copy them. SELinux works on inodes instead of paths so moving things retains the old file contexts. This was definitely a welcome to SELinux type issue. Glad it was able to resolved without anything to crazy.
Evening Newbs
Evening Newbs•5mo ago
Yeah, me too. Thanks again for all the help everyone.