Knowledge Base
-
How to move /boot from zfs on root to ext4 filesystem?
Sometimes the Logpoint (zfs on root) is not able to boot hence dropping to "grub rescue>" shell, if the system is not able to boot even after reinstalling grub, follow the below steps to boot from ext4 formatted partition which is better supported by grub2
Attach new virtual/physical disk or usb stick .eg /dev/sdx
Boot from Logpoint 6.3.0 ISO or later, get to the command line
Part 1-
# modprobe zfs# zpool import -a -N -R /mntMount the root pool into /mnt# zfs mount LP_Pool/ROOT/lp_rootmount -t proc /proc /mnt/procmount --rbind /dev /mnt/devmount --rbind /sys /mnt/syschroot /mnt /bin/bash --loginexport ZPOOL_VDEV_NAME_PATH=YESgrub-probe /# Part 2# Preparing ext4 boot disk sgdisk -p /dev/sdxsgdisk -n 1:2048:4095 -c 1:"BIOS Boot Partition" -t 1:ef02 /dev/sdx#below command will create new partition number "2" of size 3219MB sgdisk -n 2:4096:6291456 -c 2:"Linux /boot" -t 2:8300 /dev/sdxmkfs.ext4 /dev/sdx2 mv /boot /oldbootmkdir /boot
Now, mount the ext4 formatted usb disk to /boot
mount -t ext4 /dev/sdx2 /boot/rsync -aP /oldboot/* to /boot
get UUID of /dev/sdx2 and append to fstab as below:-
blkid | grep sdx2 # this will give uuid of sdx2
Edit /etc/fstab file:-
UUID=1b1d424c-9fd1-4fd2-aba7-80252f1a8d64 /boot ext4 defaults 0 2
grub-mkdevicemapgrub-install /dev/sdxupdate-grub2update-initramfs -u -k all
Exit from chroot
# zfs unmount -a# zfs set mountpoint=/ LP_Pool/ROOT/lp_root# reboot
In the bios settings make sure you select the disk just configured as a primary boot device
-
ZFS Tuning
Its recommended to maintain 20% free space on the zfs pool, storage usage of more than 80% will lead to poor IO performance and longer resilver times on raidz or mirrored pool if any disks is being replaced.
By default, ZIL or zfs intent log) lives on the same ZFS vdevs. Not only is the writes to the ZIL on rotating disks are slow but those writes and reads are competing with other disk activity this also means double writes on the same pool vdevs.
These double writes can be prevented by disabling sync on the pool, eg.
zfs set sync=disabled PoolName
this configuration poses the risk of loosing a few seconds of data if there is a sudden power loss on the server.
set xttr=sa
zfs set xattr=sa PoolName
The dedicated zil log device can improve write performance, to add a log device,
zpool add PoolName log /dev/disk/by-id/<id of ssd log disk>
Below values can be set to control the zfs arc memory (unit is bytes)
echo "9663676416" > /sys/module/zfs/parameters/zfs_arc_maxecho "1073741824" > /sys/module/zfs/parameters/zfs_arc_min
On slower storage servers the default dirty data value may be too high and can lead to kernel hung tasks timeouts. Usually on larger memory systems eg 100GB or more you can lower the dirty data value.
Default value of zfs_dirty_data_max : 10% of physical RAM, capped at zfs_dirty_data_max_max . The default value of zfs_dirty_data_max_max is 25% of physical RAM
To set 128MB as dirty data max value
echo "134217728" > /sys/module/zfs/parameters/zfs_dirty_data_max
Other tunables can be found here
-
Office 365 troubleshooting
Facing issues during the office365 fetcher configuration? In the table below we have discussed some of the frequently occurring issues and solutions.
First, make sure to open the following domains in the firewall:
- login.windows.net
- login.microsoftonline.com
- manage.office.com
S.N. Error Seen Reason for the Error 1 error=Get Token request returned http error: 401 and server response: {"error":"invalid_client","error_description":"AADSTS70002: The request body must contain the following parameter: 'client_secret or client_assertion'.\r\nTrace ID: 52eb33f4-b7d8-4b20-a987-0921e5720700\r\nCorrelation ID: c90fc381-c8c9-49ae-b04b-1dd4234a6eed\r\nTimestamp: 2017-12-15 11:09:40Z","error_codes":[70002],"timestamp":"2017-12-15 11:09:40Z","trace_id":"52eb33f4-b7d8-4b20-a987-0921e5720700","correlation_id":"c90fc381-c8c9-49ae-b04b-1dd4234a6eed"} Use of web application instead of native application Set Default Client Type to Yes Refer this 2 2017-12-18_13:56:23.11898 WARNING: exception while running job {u'http_proxy': u'', u'routing_policy': u'default', u'_enrich_policy': u'None', u'client_id': u'c04a5a8e-e41f-4463-81b8-9763df8727f6', u'o365_user_name': u'logpoint', u'tenant_id': u'5fdc6ba4-b2e4-4467-a144-0f6fe370a517', u'device_ip': u'10.100.1.146', u'device_name': u'prsnfvllg0001', u'https_proxy': u'', u'normalizer': None, u'fetch_interval': 10800, u'timezone': u'UTC', u'o365_user_password': u'@gefos2015!', u'charset': u'utf-8'}, error=Server returned an unknown AccountType: unknown Incorrect username(In this case it must have domain) 3 2018-01-21_13:57:56.77092 WARNING: An exception occured for url. Retrying: https://manage.office.com/api/v1.0/e86f4c2e-cb7f-48a6-8295-30ebee2c0abf/activity/feed/audit/20180121133014619009373$20180121133014619009373$audit_azureactivedirectory$Audit_AzureActiveDirectory. Exception: HTTPSConnectionPool(host='manage.office.com', port=443): Max retries exceeded with url: /api/v1.0/e86f4c2e-cb7f-48a6-8295-30ebee2c0abf/activity/feed/audit/20180121133014619009373$20180121133014619009373$audit_azureactivedirectory$Audit_AzureActiveDirectory?PublisherIdentifier=e86f4c2e-cb7f-48a6-8295-30ebee2c0abf (Caused by <class 'socket.error'>: [Errno 110] Connection timed out) Connection timed out 2018-01-21_13:58:00.47220 ERROR: Error: sid=office365|fetcher@logpoint.com; error='NoneType' object has no attribute 'ok' 4 2018-01-21_04:29:20.65699 WARNING: An exception occured for url. Retrying: https://manage.office.com/api/v1.0/e86f4c2e-cb7f-48a6-8295-30ebee2c0abf/activity/feed/audit/20180121033815495016684$20180121033815495016684$audit_azureactivedirectory$Audit_AzureActiveDirectory. Exception: HTTPSConnectionPool(host='manage.office.com', port=443): Read timed out. (read timeout=None) Read Time Out 2018-01-21_04:29:25.47278 ERROR: Error: sid=office365|fetcher@logpoint.com; error='NoneType' object has no attribute 'ok' 2018-01-21_10:05:29.87411 WARNING: An exception occured for url. Retrying: https://manage.office.com/api/v1.0/e86f4c2e-cb7f-48a6-8295-30ebee2c0abf/activity/feed/audit/20180121093832305015413$20180121093832305015413$audit_azureactivedirectory$Audit_AzureActiveDirectory. Exception: HTTPSConnectionPool(host='manage.office.com', port=443): Read timed out. (read timeout=None) 5 ERROR: RequestException: sid=office365|fetcher@logpoint.com; error=HTTPSConnectionPool(host='manage.office.com', port=443): Max retries exceeded with url: /api/v1.0/e86f4c2e-cb7f-48a6-8295-30ebee2c0abf/activity/feed/subscriptions/content?contentType=Audit.Exchange&endTime=2018-01-25T01%3A40%3A09&startTime=2018-01-25T00%3A40%3A09&PublisherIdentifier=e86f4c2e-cb7f-48a6-8295-30ebee2c0abf (Caused by <class 'socket.error'>: [Errno 104] Connection reset by peer) Connection reset by peer 6 ERROR: Office365Fetcher: sid=office365fetcher|e86f4c2e_cb7f_48a6_8295_30ebee2c0abf; adal_error=WS-Trust RST request returned http error: 500 and server response: <s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing"><s:Header><a:Action s:mustUnderstand="1">http://www.w3.org/2005/08/addressing/soap/fault</a:Action></s:Header><s:Body><s:Fault><s:Code><s:Value>s:Sender</s:Value><s:Subcode><s:Value xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">a:InvalidSecurity</s:Value></s:Subcode></s:Code><s:Reason><s:Text xml:lang="da-DK">An error occurred when verifying security for the message.</s:Text></s:Reason></s:Fault></s:Body></s:Envelope> Timezone mismatch between lp machine and AD machine while using federated ADFS -
Getting ADFS logs into Logpoint
You need to adjust Logpoint Agent or the NxLog configuration to get ADFS logs ingested into Logpoint.
To do this, use the <Select Path="AD FS/Admin">*</Select> tag in the NxLog or Logpoint Agent configuration.
Furthermore, you will need to add a Custom Category in LogPoint Agent as seen on the screenshot below.
-
"LDAP User/Group Mapping" in LDAP Authentication
Generally, LogPoint pulls the User and Group relationship data from the LDAP server that is being used.
Since there is a variation among LDAP server vendors on how the user-group relationship is represented, LogPoint requires data on how this mapping is done.
For instance, in case of Microsoft AD, each user will have an attribute called "memberOf" which contains all the groups the particular user belongs to.
While in case of OpenLDAP, the group will contain the attribute "member" which lists all users belonging to this group.
This enables us to configure LogPoint depending on how user-group mapping is done on the LDAP server.
For example,
- If Group in LDAP contains information about its member in a field name "myMembers" then, you need to select the "Group Contains User Info" button and input "myMembers" in the text field. In the "User Settings" section's "Group Mem attr", you will need to enter the name of the User attribute that the "myMembers" field contains.
- If User in LDAP contains information about the group it belongs to in the field "myGroups" then, you need to select the "User Contains Group info" button and input "myGroups" in the text field. In the "Group Settings" section's "Mem Group Attr", you will need to enter the name of the Group attribute that is contained by the "myGroups" field of the LDAP User.