Debugging SE Linux
Both server-1..2 and the BGFS hosts are running SE Linux in targeted mode; the SWCs are not running SE Linux although since the images are hosted on a system running SE Linux there is some slight potential that they could run afoul of SE Linux.
Most of the time SE Linux is not a problem but occasionally some strange things will occur. These usually manifest themselves as permission-denied errors. If a cursory check of the relevant file and directory permissions together with the UID in use does not show any reason for the access to be denied, it's time to see if SE Linux is the root cause.
Searching the Audit Log
The best way to go about this is to search the
audit
logs. The logs themselves live in
/var/log/audit
and are text files; however, the format is not particularly readable and the required security settings log almost everyting so the log can be very dense. To get around this the utility
ausearch
is used:
- =ausearch --message avc,user_avc [ --start [date] [time] ] [ --end [date] [time] ] =
The SE Linux denial messages will have message type "avc" which is what the first option is specifying. After that you can narrow the time range down by specifying the starting and ending date/times of interest. Date can be expressed as
m/d/yyyy
(e.g., 12/31/2019) and time can be expressed a
hh:mm:ss
using 24-hour times (e.g., 13:01:00).
Using journalctl
For more subtle problems it can be helpful to use
journalctl
. Normally, it will use a pager to display the output which is desirable unless the page truncates the output; in that case use the
--no-pager switch
. The output from
journalctl
can provide a pointer to getting more specific hints about how to address the SE Linux issue by providing an "sealert" command that can be cut and pasted into the shell (see also
Using sealert below).
- =journalctl -t setroubleshoot [ --no-pager ] [ --since yyyy-mm-dd hh:mm:ss ] [ --until yyyy-mm-dd hh:mm:ss ]
[12:51 root@S-1-T cron.daily]# journalctl -t setroubleshoot --no-pager --since "2020-09-13 03:38:01" --until "2020-09-13 03:38:04"
-- Logs begin at Mon 2020-08-31 17:14:52 EDT, end at Mon 2020-09-14 12:52:11 EDT. --
Sep 13 03:38:03 server-1.usno.nrao.edu setroubleshoot[216163]: SELinux is preventing /usr/bin/mv from write access on the directory /export/var/log/hosts/swc-001.log-20200909.gz. For complete SELinux messages run: sealert -l 944e26ff-6768-4d70-943d-8185b462e246
Sep 13 03:38:03 server-1.usno.nrao.edu setroubleshoot[216163]: SELinux is preventing /usr/bin/mv from write access on the directory /export/var/log/hosts/swc-001.log-20200913. For complete SELinux messages run: sealert -l 944e26ff-6768-4d70-943d-8185b462e246
Sep 13 03:38:03 server-1.usno.nrao.edu setroubleshoot[216163]: SELinux is preventing /usr/bin/mv from write access on the directory /export/var/log/hosts/swc-002.log-20200906.gz. For complete SELinux messages run: sealert -l 944e26ff-6768-4d70-943d-8185b462e246
Using sealert
[12:31 root@S-1-T cron.daily]# sealert -l 944e26ff-6768-4d70-943d-8185b462e246
SELinux is preventing /usr/bin/mv from write access on the directory /export/var/log/hosts.
*** Plugin catchall_labels (83.8 confidence) suggests *****************
If you want to allow mv to have write access on the hosts directory
Then you need to change the label on /export/var/log/hosts
Do
# semanage fcontext -a -t FILE_TYPE '/export/var/log/hosts'
where FILE_TYPE is one of the following: NetworkManager_log_t, abrt_var_cache_t, abrt_var_log_t, acct_data_t, afs_logfile_t, aide_log_t, amanda_log_t, antivirus_log_t, apcupsd_log_t, apmd_log_t, asterisk_log_t, auth_cache_t, bacula_log_t, bitlbee_log_t, boinc_log_t, brltty_log_t, calamaris_log_t, callweaver_log_t, canna_log_t, ccs_var_lib_t, ccs_var_log_t, certmaster_var_log_t, cfengine_log_t, cgred_log_t, checkpc_log_t, chronyd_var_log_t, cinder_log_t, cloud_log_t, cluster_var_log_t, cobbler_var_log_t, collectd_log_t, collectd_rw_content_t, condor_log_t, conman_log_t, consolekit_log_t, container_log_t, couchdb_log_t, cron_log_t, ctdbd_log_t, cupsd_log_t, cyphesis_log_t, ddclient_log_t, deltacloudd_log_t, denyhosts_var_log_t, devicekit_var_log_t, dirsrv_snmp_var_log_t, dirsrv_var_log_t, dlm_controld_var_log_t, dnsmasq_var_log_t, dovecot_var_log_t, dspam_log_t, evtchnd_var_log_t, exim_log_t, fail2ban_log_t, faillog_t, fenced_var_log_t, fetchmail_log_t, fingerd_log_t, firewalld_var_log_t, foghorn_var_log_t, fsadm_log_t, ganesha_var_log_t, getty_log_t, gfs_controld_var_log_t, glance_log_t, glusterd_log_t, groupd_var_log_t, haproxy_var_log_t, httpd_log_t, icecast_log_t, inetd_log_t, initrc_var_log_t, innd_log_t, ipa_log_t, ipsec_log_t, iscsi_log_t, iwhd_log_t, jetty_log_t, jockey_var_log_t, kadmind_log_t, keystone_log_t, kismet_log_t, krb5kdc_log_t, ksmtuned_log_t, ktalkd_log_t, lastlog_t, logrotate_tmp_t, logrotate_var_lib_t, mailman_log_t, mcelog_log_t, mdadm_log_t, minidlna_log_t, mirrormanager_log_t, mongod_log_t, motion_log_t, mpd_log_t, mrtg_log_t, munin_log_t, mysqld_log_t, mythtv_var_log_t, nagios_log_t, named_cache_t, named_log_t, neutron_log_t, nova_log_t, nscd_log_t, nsd_log_t, ntpd_log_t, numad_var_log_t, openhpid_log_t, openshift_log_t, openshift_var_lib_t, opensm_log_t, openvpn_status_t, openvpn_var_log_t, openvswitch_log_t, openwsman_log_t, osad_log_t, passenger_log_t, pcp_log_t, piranha_log_t, pkcs_slotd_log_t, pki_log_t, pki_ra_log_t, pki_tomcat_log_t, pki_tps_log_t, plymouthd_var_log_t, polipo_log_t, postgresql_log_t, pppd_log_t, pptp_log_t, prelink_log_t, prelude_log_t, privoxy_log_t, procmail_log_t, prosody_log_t, psad_var_log_t, puppet_log_t, pyicqt_log_t, qdiskd_var_log_t, rabbitmq_var_log_t, radiusd_log_t, redis_log_t, rhev_agentd_log_t, rhsmcertd_log_t, ricci_modcluster_var_log_t, ricci_var_log_t, rpm_log_t, rsync_log_t, rtas_errd_log_t, samba_log_t, sanlock_log_t, sectool_var_log_t, sendmail_log_t, sensord_log_t, setroubleshoot_var_log_t, shorewall_log_t, slapd_log_t, slpd_log_t, smsd_log_t, snapperd_log_t, snmpd_log_t, snort_log_t, spamd_log_t, speech-dispatcher_log_t, squid_log_t, sssd_var_log_t, stapserver_log_t, stunnel_log_t, sudo_log_t, svnserve_log_t, sysstat_log_t, systemd_passwd_var_run_t, thin_aeolus_configserver_log_t, thin_log_t, tmp_t, tomcat_log_t, tor_var_log_t, tuned_log_t, ulogd_var_log_t, uucpd_log_t, var_lib_t, var_lock_t, var_log_t, var_spool_t, varnishlog_log_t, vdagent_log_t, virt_cache_t, virt_log_t, virt_qemu_ga_log_t, vmware_log_t, watchdog_log_t, winbind_log_t, wtmp_t, xdm_log_t, xend_var_log_t, xenstored_var_log_t, xferlog_t, xserver_log_t, zabbix_log_t, zarafa_deliver_log_t, zarafa_gateway_log_t, zarafa_ical_log_t, zarafa_indexer_log_t, zarafa_monitor_log_t, zarafa_server_log_t, zarafa_spooler_log_t, zebra_log_t, zoneminder_log_t.
Then execute:
restorecon -v '/export/var/log/hosts'
*** Plugin catchall (17.1 confidence) suggests ************************
If you believe that mv should be allowed write access on the hosts directory by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'mv' --raw | audit2allow -M my-mv
# semodule -i my-mv.pp
Additional Information:
Source Context system_u:system_r:logrotate_t:s0-s0:c0.c1023
Target Context unconfined_u:object_r:usr_t:s0
Target Objects /export/var/log/hosts [ dir ]
Source mv
Source Path /usr/bin/mv
Port
Host server-1.usno.nrao.edu
Source RPM Packages
Target RPM Packages
Policy RPM selinux-policy-3.13.1-266.el7_8.1.noarch
Selinux Enabled True
Policy Type targeted
Enforcing Mode Enforcing
Host Name server-1.usno.nrao.edu
Platform Linux server-1.usno.nrao.edu
-
-
-
-
-
-
-
-
-
- 10.0-1127.13.1.el7.x86_64 #1 SMP Fri Jun 12
- :34:17 EDT 2020 x86_64 x86_64
Alert Count 80
First Seen 2020-08-09 03:20:04 EDT
Last Seen 2020-09-13 03:38:01 EDT
Local ID 944e26ff-6768-4d70-943d-8185b462e246
Raw Audit Messages
type=AVC msg=audit(1599982681.772:82570): avc: denied { write } for pid=216186 comm="mv" name="hosts" dev="md1" ino=28049411 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:usr_t:s0 tclass=dir permissive=0
Hash: mv,logrotate_t,usr_t,dir,write
Ways to Run Afoul of SE Linux
SE Linux tends to stay out of the way, most of the time. However, it does treat system-related files specially. Under SE Linux files are tagged with a security context. The context for a file can be seen by using the "-Z" switch to ls
:
[11:50 root@S-1-T hosts]# ls -Z /etc/passwd
-rw-r--r--. root root system_u:object_r:passwd_file_t:s0 /etc/passwd
The passwd
file above is shown to be associated with the "system_u" user and be of type "passwd_file_t"; the other two context components, object_r
and s0
, tend not to be too important in the SE Linux=targeted= mode. Usually, it's the file type that causes problems under SE Linux. Because this file has a special type, SE Linux has a number of rules that it enforces on what can be done to this file and it will deny some operations. Conversely, system programs such as useradd
will refuse to operate on files that are not properly labelled.
Fixing Problems
One problem that cropped up during system installation was caused by copying/moving the password-related files (/etc/{passwd,group,shadow,gshadow} around. This resulted in the file in the systems /etc
directory not being labelled correctly which prevented log in and the use of account management utilities. The easiest fix is to do restorecon file(s)
which restores context labels files having the appropiate name and location.
Other issues tend to require the hints found in the journalctl
and sealert
output shown above. In particular, it might be necessary to enable an SE Linux boolean to permit a particular type of access which is denied by default. For example, a boolean must be set to allow the user of NFS mounted home directories (this was on the BGFS hosts):
-
sesetbool -P use_nfs_home_dirs 1
- Allows the use of NFS mounted home directories (-P makes the setting permanent).
To see what booleans have been changed use semanage booleans --list --locallist
which lists those booleans that differ from the standard SE Linux policy.
[13:09 root@S-1-p admin]# ssh bg-mds-1 semanage boolean --list --locallist
SELinux boolean State Default Description
use_nfs_home_dirs (on , on) Allow use to nfs home dirs
- JimJacobs - 2020-09-14