!SLIDE subsectionnonum # Troubleshooting !SLIDE smbullets small # Logging * Foreman * Increase log level in general * Enable additional specific logs * Smart proxy * Increase log level * Smart proxies log can be read on the WebGUI * Plugins can add their own logging ~~~SECTION:handouts~~~ **** Both Foreman and the Smart proxy write an excellent log file. Foreman log files are located in "/var/log/foreman" and its main log is called "production.log". You can find all accessed URLs and errors generated by accessing it. If the standard log level does not provide any information it is possible to increase the log level by changing the configuration in the "/etc/foreman/settings.yaml". :logging: :level: debug Furthermore it is possible to enable additional specific loggers. * app - web requests and all general application logs (default: true) * ldap - high level LDAP queries (e.g. find users in group) and LDAP operations performed (default: false) * permissions - evaluation of user roles, filters and permissions when loading pages (default: false) * sql - SQL queries made through Rails ActiveRecord, only debug (default: false) For those add following to the configuration: :loggers: :ldap: :enabled: true The log files of the Smart proxy are located in "/var/log/foreman-proxy" containing all errors issued by the Smart proxy like downloading of boot images failed and failed updates of services. If additional details are needed increase the log level in "/etc/foreman-proxy/settings.yml". :log_level: DEBUG Plugins to both components can add directly to the log or write their own, so have a look into their configuration if attempting any troubleshooting with a specific plugin. You can enable Logs access from the WebGUI by adding in "/etc/foreman-proxy/settings.d/logs.yml": :enabled: https ~~~ENDSECTION~~~ !SLIDE smbullets small # Provisioning process * Watch the output in the WebGUI * Spoof the provisioning configuration * Find the communication in Foreman log * In the Smart proxy log * Download of boot images * Configuration changes failing * On the TFTP server * Checksum of boot image * PXE configuration * Reports of configuration management ~~~SECTION:handouts~~~ **** Troubleshooting of the provisioning process can be required if deployment of a system fails or differs from expectation. In most cases the output of the WebGUI is quite useful, but sometimes a deeper look is required to find a solution. In addition to the template preview at the host you can get the provisioning configuration files by spoofing the URL: https://foreman.localdomain/unattended/provision?spoof=10.0.0.100 In this URL you can change the template type provision to every other type to get those files. If everything here seems to be correct, check the communication in the Foreman log. Getting the exact URLs from the log and browsing them can help to find issues like token on a boot disk expired. Next step would be to look for errors in the Smart proxy log if the download of the boot images or some configuration changes issued to the managed services have failed. On the TFTP server have a look at the boot image, it is downloaded only once during the first deployment of a host started. If somehow the image was already downloaded you will not find any new log entry about the failed download. Also in some rare cases distribution releases a newer version of the boot image to fix some installation issues. So check the size and checksum of the image if the boot fails. ~~~PAGEBREAK~~~ In the case there is no PXE configuration you can cancel and restart the build process or execute a rebuild from the "All Hosts" menu. If the deployment of the system works but something is not configured in the expected way have a look in the reports of the configuration management. In many cases Puppet code is not well designed or has some cross system dependencies so the agent has to run more than once to complete the full configuration. ~~~ENDSECTION~~~ !SLIDE smbullets small # Upstream support * Common issues in the Wiki * IRC * User channel for support * Developer channel for development support * Community Platform based on Discourse * Server fault * Youtube * Issue tracker * Gathering information * Debug tool (to be deprecated) * sosreport ~~~SECTION:handouts~~~ **** The Foreman team tries to help with issues as good as they can. To achieve this there are several ways to get support available. First have a look into the Wiki page collecting common issues: http://projects.theforeman.org/projects/1/wiki/Troubleshooting If your problem is not covered there the fastest way to get in touch with the team is the IRC Channel. The channel #theforeman is hosted on irc.freenode.net, the developer channel #theforeman-dev is only for support on development of Foreman and plugins. The mailing lists have been replaced with Discourse to provide a Community Platform which is basicly a forum with mailing list support. You can get help here, discuss development and follow release workflow including test weeks. On 'Server Fault' you can ask questions and find answers tagged with foreman: http://serverfault.com/questions/tagged/foreman On Youtube in the Foreman channel you can find introductions, deep dives, community demos and other ressources which can also be very useful: https://www.youtube.com/channel/UCCo7AZ1oG6TbG0-dwjRqCmw Looking at all those resources will solve most issues, if not it is probably a bug, so search for it in the issue tracker and if you can not find a similar issue file create a new one with as much information as possible to help fixing the bug and others with the same issue. http://projects.theforeman.org/projects/foreman/issues Sometimes you could be asked to send a debug report which could be generated with "foreman-debug" including the complete configuration stripped of security data. With "foreman-debug -u" it could be uploaded to a location only accessable by the Foreman core developers. This tool is going to be deprecated and be replaced by a plugin for sosreport. Sosreport can collect even more system information and based other already existing plugins also those of required or managed services. The only feature missing is the built-in upload of the report. ~~~ENDSECTION~~~