Debugging SE Linux

Both server-1..2 and the BGFS hosts are running SE Linux in targeted mode; the SWCs are not running SE Linux although since the images are hosted on a system running SE Linux there is some slight potential that they could run afoul of SE Linux.

Most of the time SE Linux is not a problem but occasionally some strange things will occur. These usually manifest themselves as permission-denied errors. If a cursory check of the relevant file and directory permissions together with the UID in use does not show any reason for the access to be denied, it's time to see if SE Linux is the root cause.

Searching the Audit Log

The best way to go about this is to search the audit logs. The logs themselves live in /var/log/audit and are text files; however, the format is not particularly readable and the required security settings log almost everyting so the log can be very dense. To get around this the utility ausearch is used:

  • =ausearch --message avc,user_avc [ --start [date] [time] ] [ --end [date] [time] ] =

The SE Linux denial messages will have message type "avc" which is what the first option is specifying. After that you can narrow the time range down by specifying the starting and ending date/times of interest. Date can be expressed as m/d/yyyy (e.g., 12/31/2019) and time can be expressed a hh:mm:ss using 24-hour times (e.g., 13:01:00).

More Tools

Using journalctl

For more subtle problems it can be helpful to use journalctl. Normally, it will use a pager to display the output which is desirable unless the page truncates the output; in that case use the --no-pager switch. The output from journalctl can provide a pointer to getting more specific hints about how to address the SE Linux issue by providing an "sealert" command that can be cut and pasted into the shell (see also Using sealert below).

  • =journalctl -t setroubleshoot [ --no-pager ] [ --since yyyy-mm-dd hh:mm:ss ] [ --until yyyy-mm-dd hh:mm:ss ]
[12:51 root@S-1-T cron.daily]# journalctl -t setroubleshoot --no-pager --since "2020-09-13 03:38:01" --until "2020-09-13 03:38:04"
-- Logs begin at Mon 2020-08-31 17:14:52 EDT, end at Mon 2020-09-14 12:52:11 EDT. --
Sep 13 03:38:03 server-1.usno.nrao.edu setroubleshoot[216163]: SELinux is preventing /usr/bin/mv from write access on the directory /export/var/log/hosts/swc-001.log-20200909.gz. For complete SELinux messages run: sealert -l 944e26ff-6768-4d70-943d-8185b462e246
Sep 13 03:38:03 server-1.usno.nrao.edu setroubleshoot[216163]: SELinux is preventing /usr/bin/mv from write access on the directory /export/var/log/hosts/swc-001.log-20200913. For complete SELinux messages run: sealert -l 944e26ff-6768-4d70-943d-8185b462e246
Sep 13 03:38:03 server-1.usno.nrao.edu setroubleshoot[216163]: SELinux is preventing /usr/bin/mv from write access on the directory /export/var/log/hosts/swc-002.log-20200906.gz. For complete SELinux messages run: sealert -l 944e26ff-6768-4d70-943d-8185b462e246

Using sealert

[12:31 root@S-1-T cron.daily]# sealert -l 944e26ff-6768-4d70-943d-8185b462e246 SELinux is preventing /usr/bin/mv from write access on the directory /export/var/log/hosts.

*** Plugin catchall_labels (83.8 confidence) suggests *****************

If you want to allow mv to have write access on the hosts directory Then you need to change the label on /export/var/log/hosts Do # semanage fcontext -a -t FILE_TYPE '/export/var/log/hosts' where FILE_TYPE is one of the following: NetworkManager_log_t, abrt_var_cache_t, abrt_var_log_t, acct_data_t, afs_logfile_t, aide_log_t, amanda_log_t, antivirus_log_t, apcupsd_log_t, apmd_log_t, asterisk_log_t, auth_cache_t, bacula_log_t, bitlbee_log_t, boinc_log_t, brltty_log_t, calamaris_log_t, callweaver_log_t, canna_log_t, ccs_var_lib_t, ccs_var_log_t, certmaster_var_log_t, cfengine_log_t, cgred_log_t, checkpc_log_t, chronyd_var_log_t, cinder_log_t, cloud_log_t, cluster_var_log_t, cobbler_var_log_t, collectd_log_t, collectd_rw_content_t, condor_log_t, conman_log_t, consolekit_log_t, container_log_t, couchdb_log_t, cron_log_t, ctdbd_log_t, cupsd_log_t, cyphesis_log_t, ddclient_log_t, deltacloudd_log_t, denyhosts_var_log_t, devicekit_var_log_t, dirsrv_snmp_var_log_t, dirsrv_var_log_t, dlm_controld_var_log_t, dnsmasq_var_log_t, dovecot_var_log_t, dspam_log_t, evtchnd_var_log_t, exim_log_t, fail2ban_log_t, faillog_t, fenced_var_log_t, fetchmail_log_t, fingerd_log_t, firewalld_var_log_t, foghorn_var_log_t, fsadm_log_t, ganesha_var_log_t, getty_log_t, gfs_controld_var_log_t, glance_log_t, glusterd_log_t, groupd_var_log_t, haproxy_var_log_t, httpd_log_t, icecast_log_t, inetd_log_t, initrc_var_log_t, innd_log_t, ipa_log_t, ipsec_log_t, iscsi_log_t, iwhd_log_t, jetty_log_t, jockey_var_log_t, kadmind_log_t, keystone_log_t, kismet_log_t, krb5kdc_log_t, ksmtuned_log_t, ktalkd_log_t, lastlog_t, logrotate_tmp_t, logrotate_var_lib_t, mailman_log_t, mcelog_log_t, mdadm_log_t, minidlna_log_t, mirrormanager_log_t, mongod_log_t, motion_log_t, mpd_log_t, mrtg_log_t, munin_log_t, mysqld_log_t, mythtv_var_log_t, nagios_log_t, named_cache_t, named_log_t, neutron_log_t, nova_log_t, nscd_log_t, nsd_log_t, ntpd_log_t, numad_var_log_t, openhpid_log_t, openshift_log_t, openshift_var_lib_t, opensm_log_t, openvpn_status_t, openvpn_var_log_t, openvswitch_log_t, openwsman_log_t, osad_log_t, passenger_log_t, pcp_log_t, piranha_log_t, pkcs_slotd_log_t, pki_log_t, pki_ra_log_t, pki_tomcat_log_t, pki_tps_log_t, plymouthd_var_log_t, polipo_log_t, postgresql_log_t, pppd_log_t, pptp_log_t, prelink_log_t, prelude_log_t, privoxy_log_t, procmail_log_t, prosody_log_t, psad_var_log_t, puppet_log_t, pyicqt_log_t, qdiskd_var_log_t, rabbitmq_var_log_t, radiusd_log_t, redis_log_t, rhev_agentd_log_t, rhsmcertd_log_t, ricci_modcluster_var_log_t, ricci_var_log_t, rpm_log_t, rsync_log_t, rtas_errd_log_t, samba_log_t, sanlock_log_t, sectool_var_log_t, sendmail_log_t, sensord_log_t, setroubleshoot_var_log_t, shorewall_log_t, slapd_log_t, slpd_log_t, smsd_log_t, snapperd_log_t, snmpd_log_t, snort_log_t, spamd_log_t, speech-dispatcher_log_t, squid_log_t, sssd_var_log_t, stapserver_log_t, stunnel_log_t, sudo_log_t, svnserve_log_t, sysstat_log_t, systemd_passwd_var_run_t, thin_aeolus_configserver_log_t, thin_log_t, tmp_t, tomcat_log_t, tor_var_log_t, tuned_log_t, ulogd_var_log_t, uucpd_log_t, var_lib_t, var_lock_t, var_log_t, var_spool_t, varnishlog_log_t, vdagent_log_t, virt_cache_t, virt_log_t, virt_qemu_ga_log_t, vmware_log_t, watchdog_log_t, winbind_log_t, wtmp_t, xdm_log_t, xend_var_log_t, xenstored_var_log_t, xferlog_t, xserver_log_t, zabbix_log_t, zarafa_deliver_log_t, zarafa_gateway_log_t, zarafa_ical_log_t, zarafa_indexer_log_t, zarafa_monitor_log_t, zarafa_server_log_t, zarafa_spooler_log_t, zebra_log_t, zoneminder_log_t. Then execute: restorecon -v '/export/var/log/hosts'

*** Plugin catchall (17.1 confidence) suggests ************************

If you believe that mv should be allowed write access on the hosts directory by default. Then you should report this as a bug. You can generate a local policy module to allow this access. Do allow this access for now by executing: # ausearch -c 'mv' --raw | audit2allow -M my-mv # semodule -i my-mv.pp

Additional Information: Source Context system_u:system_r:logrotate_t:s0-s0:c0.c1023 Target Context unconfined_u:object_r:usr_t:s0 Target Objects /export/var/log/hosts [ dir ] Source mv Source Path /usr/bin/mv Port Host server-1.usno.nrao.edu Source RPM Packages Target RPM Packages Policy RPM selinux-policy-3.13.1-266.el7_8.1.noarch Selinux Enabled True Policy Type targeted Enforcing Mode Enforcing Host Name server-1.usno.nrao.edu Platform Linux server-1.usno.nrao.edu
                    1. 10.0-1127.13.1.el7.x86_64 #1 SMP Fri Jun 12
                    2. :34:17 EDT 2020 x86_64 x86_64
Alert Count 80 First Seen 2020-08-09 03:20:04 EDT Last Seen 2020-09-13 03:38:01 EDT Local ID 944e26ff-6768-4d70-943d-8185b462e246

Raw Audit Messages type=AVC msg=audit(1599982681.772:82570): avc: denied { write } for pid=216186 comm="mv" name="hosts" dev="md1" ino=28049411 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:usr_t:s0 tclass=dir permissive=0

Hash: mv,logrotate_t,usr_t,dir,write

Ways to Run Afoul of SE Linux

SE Linux tends to stay out of the way, most of the time. However, it does treat system-related files specially. Under SE Linux files are tagged with a security context. The context for a file can be seen by using the "-Z" switch to ls:

[11:50 root@S-1-T hosts]# ls -Z /etc/passwd
-rw-r--r--. root root system_u:object_r:passwd_file_t:s0 /etc/passwd

The passwd file above is shown to be associated with the "system_u" user and be of type "passwd_file_t"; the other two context components, object_r and s0, tend not to be too important in the SE Linux=targeted= mode. Usually, it's the file type that causes problems under SE Linux. Because this file has a special type, SE Linux has a number of rules that it enforces on what can be done to this file and it will deny some operations. Conversely, system programs such as useradd will refuse to operate on files that are not properly labelled.

Fixing Problems

One problem that cropped up during system installation was caused by copying/moving the password-related files (/etc/{passwd,group,shadow,gshadow} around. This resulted in the file in the systems /etc directory not being labelled correctly which prevented log in and the use of account management utilities. The easiest fix is to do restorecon file(s) which restores context labels files having the appropiate name and location.

Other issues tend to require the hints found in the journalctl and sealert output shown above. In particular, it might be necessary to enable an SE Linux boolean to permit a particular type of access which is denied by default. For example, a boolean must be set to allow the user of NFS mounted home directories (this was on the BGFS hosts):

sesetbool -P use_nfs_home_dirs 1
Allows the use of NFS mounted home directories (-P makes the setting permanent).

To see what booleans have been changed use semanage booleans --list --locallist which lists those booleans that differ from the standard SE Linux policy.

[13:09 root@S-1-p admin]# ssh bg-mds-1 semanage boolean --list --locallist
SELinux boolean                State  Default Description
use_nfs_home_dirs              (on   ,   on)  Allow use to nfs home dirs

- JimJacobs - 2020-09-14
Topic revision: r2 - 2020-09-14, JimJacobs
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback