Read-Only file system Check Point / Linux

You may experience issues like a Check Point installation not being responsive (processes not started, etc.) and your dmesg, messages, $FWDIR/log/fwd.elg and fwm.elg may be filled with all sorts of error messages indicating the lack of files or not being able to create files. In general it behaves in such a way you’ll think “I bloody well hope I did backup the system recently”.

Underneath you’ll find some of the errors I got when experiencing this problem.
Please note; if your system has _not_ entered a read-only state and you are looking to resolve some of the issues below, you would be best off continuing your search. =)

 

Conclusion (for those of you seeking the quick answer)

The /var-partition has entered a read-only state and had to be subjected to a manual file system check (fsck) in single user mode before allowing it to be set to read-write.

A small disclaimer; this post is written by a linux amateur, so there is probably some crucial theory left out and other, and perhaps better, ways to rectify similiar situations. =)
Use this knowledge at your own risk. ,)

Some of the error messages you may get in your general Check Point logs
CPPRODIS_init_error_logging_ex: failed to set cyclic log file for product ‘cpshared’ application ‘PostgreSQLCmd’ file ‘/opt/CPshrd-R75.20/log/postgresqlcmd.elg’.
cvpnd_admin: Can opening TdError log file /opt/CPcvpn-R75.20/log/CvpndAdmin.log: Unknown error 4294967295
cvpnd_admin: ndb_open : failed for /opt/CPsuite-R75.20/fw1/database/fwauth.NDB: Unknown error 4294967295
cvpnd_admin: Users Database is lost: unable to reload

Entries in boot.log
bp_init: FATAL: Module bpctl_mod not found.
bp_init: chgrp: failed to get attributes of `/dev/bpctl0′: No such file or directory
bp_init: chmod: failed to get attributes of `/dev/bpctl0′: No such file or directory

Entries in messages
bp_init: FATAL: Module bpctl_mod not found.
bp_init: chmod: failed to get attributes of `/dev/bpctl0′: No such file or directory
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.

Entries in dmesg
Aborting journal on device sda7
ext3_abort called
EXT3-fs error (device sda7): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only

Why does this happen?
From what I have gathered, any linux installation will set its file system in a read-only mode if there is some indication of hard drive malfunction or some issues with regards to SAN-configuration.
It supposedly does this in order to prevent data corruption, which is neat I am sure.
In this particular situation it seems there is some degree of corruption already, so the file system is re-mounted in read-only.
Although the entries in messages says recovery is completed, you can see from the /proc/mounts wether or not a partition is indeed read-write-enabled:

 [Expert@MyAwesomeChkpMgmtbox]# cat /proc/mounts
 rootfs / rootfs rw 0 0
 /dev/root / ext3 rw,data=ordered 0 0
 /dev /dev tmpfs rw 0 0
 /proc /proc proc rw 0 0
 /sys /sys sysfs rw 0 0
 /proc/bus/usb /proc/bus/usb usbfs rw 0 0
 none /dev/pts devpts rw 0 0
 /dev/sda1 /boot ext3 rw,data=ordered 0 0
 /dev/sda5 /opt ext3 rw,data=ordered 0 0
 /dev/sda2 /sysimg ext3 rw,data=ordered 0 0
 /dev/sda7 /var ext3 ro ,data=ordered 0 0

One solution is to re-mount the /var partition as a read-write-enabled partition using one of the following commands, but it did not sort out my issue as it returned an error.

[Expert@MyAwesomeChkpMgmtbox]# mount -o remount /var
mount: block device /dev/sda7 is write-protected, mounting read-only
[Expert@MyAwesomeChkpMgmtbox]# mount -t ext3 -o rw,remount /dev/sda7 /var
mount: block device /dev/sda7 is write-protected, mounting read-only

 

The solution

In order to resolve this issue, I ran fsck in maintenance mode, which resolved the issues and allowed the /var-partition to be mounted rw.

Enter boot menu
Press key to see boot menu

 

Enter maintenance mode
Choose maintenance mode and enter your expert password

 

I had to unmount the partition using umount /var before running the file check;
Running fsck
fsck -fvc /dev/sda7
The time it takes to complete depends on your partition size, but for my 60-70GB it took approximately 30-45minutes.
And once the test is completed, you can start rectifying the problem:
Problems found, which can be fixed
Allowing the utility to fix the issue seemed like a good idea.
Note – there is a switch in fsck which allows it to fix any problems without prompting for your confirmation.

Once the utility has been allowed to sort out the issue it found it presents a summary of your file system. At this point I rebooted the machine and everything was sorted.
File system summary
 

Hope this helps

5.00 avg. rating (99% score) - 1 vote

2 Responses to Read-Only file system Check Point / Linux

Leave a Reply

Your email address will not be published. Required fields are marked *