Thursday, November 06, 2008

Nim commands for client OS installation

nim -o reset -F machine_name
nim -o allocate -a spot=spot_aix53 machine_name
nim -o allocate -a source=source_aix53 machine_name
nim -o allocate -a mksysb=mksysb_5300_08 machine_name
nim -o bos_inst -a source=mksysb -a boot_client=no -a installp_flags=-agX machine_name

Tuesday, July 24, 2007

Debugging on Solaris

About Crash Dump File:
unix.<x> - Symbol Tables
vmcore.<x> - Memory Dump
bounds - Contains the sequence number to use for the next execution of savecore

Debugging with MDB

Debugging with SCAT (SUNWscat)
By default Solaris CAT is started in Read-only mode but it can be started with write privileges by using '--write' flag.

Command: dumpinfo
Description: Provides brief overview of crash dump.
How to use: /opt/SUNWscat/bin/dumpinfo ./vmcore.0
Sample Output:
Dump_panicstring="sync initiated"

Command: dumphdr
Description: Provides information about crash header.
How to use: /opt/SUNWscat/bin/dumphdr ./vmcore.0
Sample Output:
magic: defec8ed
version: 9
wordsize: 64
start: 215220224
ksyms: 8192
pfn: 925696
map: 1007616
data: 1662976
utsname.sysname: SunOS
utsname.nodename: sunultrafour.delhi.solidcore.local
utsname.release: 5.8
utsname.version: Generic_108528-29
utsname.machine: sun4u
platform: SUNW,Ultra-60
panicstr: sync initiated
crashtime: Tue Jul 24 17:21:02 EDT 2007
pageshift: 13
pagesize: 8192 (8K)
hashmask: 0x3fff
nvtop: 9821
npages: 9812 (76.0M)
ksyms_size: 912693 (891K)
ksyms_csize: 912693 (891K)
calc/actual size: 82042880 (78.0M)

How to invoke scat?
  • Debugging crash dump: /opt/SUNWscat/bin/scat <number of crash> [Example for vmcore.0 it will be <0>]. This will provide SolarisCAT prompt. From here rest of the commands are used.
  • Debugging running kernel: /opt/SUNWscat/bin/scat. Will provide SolarisCAT(live)> similar kind of prompt

How to view stack of panic thread?
'thread' or 'panic' provides stack trace of thread which was on CPU at the time of crash or the thread that initiated crash dump.

How to view stack of all the threads?
'stack summary' provides a concise stack of all the threads in the system at the time of crash.
'tlist' provides detailed stack with pid, registers, function parameters, cpu, etc etc.

How to see 'msgbuf'?

How to dump value of some global variable or of some memory address?
'rd' reads one word from memory. On 32-bit Machine 4 bytes and on 64-bit machine 8 bytes.
'rd32' reads 4 bytes of data.
'rd64' reads 8 bytes of data.
'rds' shows string data.
'rdh' dumps half word of memory (rdh <addr> <count>)

rd32 lwp_default_stksize [output: 0x10470394 genunix(bss):lwp_default_stksize+0x0 = 0x4000]
rd rootvp [output: 0x1046e238 genunix(bss):rootvp+0x0 = 0x30000ed9e10]

How to dump memory at some address and show corresponding structure?

sdump 0x1046e238 vnode - Prints vnode structure for rootvp

v_lock = {
_opaque = [ 0x30000ed9e10 ]
v_flag = 0x0
v_count = 0x166000
v_vfsmountedhere = 0x30000067508
v_op = NULL
v_vfsp = 0x30000085ce8
v_stream = NULL
v_pages = NULL
v_type = 0 (VNON)
v_rdev = 0x1500000018 (21(log),24)
v_data = 0xe00000003
v_filocks = 0x1800000007
v_shrlocks = 0x6b00000000
v_cv = {
_opaque = 0x0
v_locality = 0x4a4

How to see fields of a structure?

stype vnode

struct vnode { (size: 0x70 bytes)
typedef kmutex_t = struct mutex { (size: 0x8 bytes)
void *[0x1] _opaque; (offset 0x0 bytes, size 0x8 bytes)
} v_lock; (offset 0x0 bytes, size 0x8 bytes)
typedef ushort_t = unsigned short v_flag; (offset 0x8 bytes, size 0x2 bytes)
typedef uint_t = unsigned int v_count; (offset 0xc bytes, size 0x4 bytes)
struct vfs *v_vfsmountedhere; (offset 0x10 bytes, size 0x8 bytes)
struct vnodeops *v_op; (offset 0x18 bytes, size 0x8 bytes)
struct vfs *v_vfsp; (offset 0x20 bytes, size 0x8 bytes)
struct stdata *v_stream; (offset 0x28 bytes, size 0x8 bytes)
struct page *v_pages; (offset 0x30 bytes, size 0x8 bytes)
enum vtype {VNON=0, VREG=1, VDIR=2, VBLK=3, VCHR=4, VLNK=5, VFIFO=6, VDOOR=7, VPROC=8, VSOCK=9, VBAD=10} v_type; (offset 0x38 bytes, size 0x4 bytes)
typedef dev_t = unsigned long/long long v_rdev; (offset 0x40 bytes, size 0x8 bytes)
typedef caddr_t = char *v_data; (offset 0x48 bytes, size 0x8 bytes)
struct filock *v_filocks; (offset 0x50 bytes, size 0x8 bytes)
struct shrlocklist *v_shrlocks; (offset 0x58 bytes, size 0x8 bytes)
typedef kcondvar_t = struct _kcondvar { (size: 0x2 bytes)
typedef ushort_t = unsigned short _opaque; (offset 0x60 bytes, size 0x2 bytes)
} v_cv; (offset 0x60 bytes, size 0x2 bytes)
void *v_locality; (offset 0x68 bytes, size 0x8 bytes)
} ;

How to get assembly instruction for some address?
"rdi" dumps memory in instruction format starting for <addr> for the specified <count>. If option
  • '-f' - the entire function that the <addr> is in displayed.
  • '-b' - the entire function up to <addr> is displayed.
  • '-m' - specified modules entire text segment is displayed.
  • '-p' - <count> instructions before <addr> are displayed.

Writing Kernel Memory Variables on the fly?
'scat --write' (opens kernel with write privileges)

'wr' - write word (example: wr memscrub_verbose 1)
'wr hword' - Half word (wr hword memscrub_verbose 1)

Querying size of some type?
calc "sizeof(<data type>)" - example calc "sizeof(uint_t)"

How to see frame's data?
'frame <pc>'
Lets consider following stack trace:
<leaf trap>genunix:strcmp+0x8 (%o:0x7813b888, 0x0, 0x3c, 0x2, 0x2, 0x10436800)
calkm:sc_getvfsswbyname+0x88 (0x10436b08, 0x27, 0x7813b9e0, 0x7813b8c8, 0x780e4000, 0x0)
calkm:hook_vfsops+0xf8 (0x780e4090, 0x0, 0x780e4088, 0x7813b800, 0x7813b800, 0x7813b800)
calkm:sc_dwrite_init+0xf0 (0x5, 0x10432298, 0x10434298, 0x78129900, 0x780e4000, 0x0)
calkm:_init+0xa0 (0x0, 0x7811a000, 0x0, 0x0, 0x30001efaf98, 0x300000328b0)
genunix:modinstall+0x144 (0x10437c00, 0x40000000, 0x300021c2820, 0x7811a000, 0x26, 0x0)
genunix:mod_hold_installed_mod+0x4c (0x300021c2820, 0x2a1002f1a3c, 0x0, 0x0, 0x300021c2820, 0x0)

'frame sc_getvfsswbyname' will provide data for this frame
'frame hook_vfsops' will provide data for next frame.

How to dump data starting at some memory location?
'mdump' - example 'mdump <addr> <count>

How to search for all possible stacks?
'findstk' - finds all possible stacks
  • '-m' - reports only stack containing module
  • '-c' - reports only stack containing function

Web Links:

Thursday, June 14, 2007

Solaris 9 not booting?
/etc/system does not exist?
Driver crashing while booting?
System not booting in single user mode? .............. If answer is YES, go ahead and read

An interactive boot (boot -a) stops and asks for input during the boot process. The system provides a dialog box in which it displays the default boot values and gives you the option of changing them. You might want to boot interactively to make a temporary change to the system file or kernel. Booting interactively enables you to test your changes and recover easily if you have problems. To do this, follow the process in Step by Step 3.1.

Step by Step 3.1: The Interactive Boot Process

  1. At the ok prompt, type boot -a and press Enter. The boot program prompts you interactively.

  2. Press Enter to use the default kernel (/kernel/unix) as prompted, or type the name of the kernel to use for booting and then press Enter.

  3. Press Enter to use the default modules directory path as prompted, or type the path for the modules directory and then press Enter.

  4. Press Enter to use the default /etc/system file as prompted, or type the name of the system file and then press Enter.

  5. Press Enter to use the default root file system type as prompted (that is, ufs for local disk booting or nfs for diskless clients).

  6. Press Enter to use the default physical name of the root device as prompted or type the device name.


The Interactive Boot Process For the exam, you should make sure you understand what each step of an interactive boot process is asking for. For example, you should know the name of the default kernel, know what the default modules are and where they are located, understand what the /etc/system file is used for, and what is meant by the default root file system. Each of these are described in the section "The Kernel," later in this chapter.

The following output shows an example of an interactive boot session:

ok boot -a
Boot device: /pci@1f,0/pci@1,1/ide@3/disk@0,0:a File and args: -a
Enter filename [kernel/sparcv9/unix]:
Enter default directory for modules [/platform/SUNW,Ultra-5_10/kernel
/platform/sun4u/kernel /kernel /usr/kernel]:
Name of system file [etc/system]:
SunOS Release 5.9 Version Generic_112233-02 64-bit
Copyright 1983-2002 Sun Microsystems, Inc. All rights reserved.
root filesystem type [ufs]:
Enter physical name of root device
configuring IPv4 interfaces: hme0.
configuring IPv6 interfaces: hme0.
Hostname: ultra5

The system is coming up. Please wait.
checking ufs filesystems
/dev/rdsk/c0t0d0s5: is clean.
/dev/rdsk/c0t0d0s7: is clean.
/dev/rdsk/c0t0d0s6: is clean.
Starting IPv6 neighbor discovery.
Setting default IPv6 interface for multicast: add net ff00::/8: \
gateway ultra5
Starting rpc services: rpcbind done.
Setting default IPv4 interface for multicast: add net 224.0/4:

Print services started.
volume management starting.
Mar 13 09:21:48
The system is ready.
ultra5 console login:


A Missing /etc/system File If the /etc/system file is missing at bootup, you see this message:

Warning cannot open system file!

The system still boots, however, using all "default" kernel parameters. Because by default the lines in the /etc/system file are all commented by the asterisk (*) character, /etc/system is actually an "empty" file. The kernel doesn't use anything from this file until you edit this file and enter an uncommented line. You can specify /dev/null (an empty file) for the system filename, and the system still boots. In fact, if the /etc/system file gets corrupted and the system won't boot from the /etc/system/dev/null to get the system to boot. file, you can specify a file named

Thursday, May 17, 2007

Workaround for "FATAL: system is not bootable, boot command is disabled" on an obp

Poor error messages are a major source of annoyance. I hit this one today, for the first time in a few years. Background info - a v210 was rather abruptly powered down and feeling somewhat ill. So I logged onto the sc and got to my console, and type boot as one does.
{1} ok boot
FATAL: system is not bootable, boot command is disabled
Which is about as helpful as someone telling me the box is currently a brick. Which I know already. Anyway just in case you happen to hit this the fix/workaround is to set auto-boot? to false, reset the box, and then set it to true and finally boot as shown below.
{1} ok setenv auto-boot? false
auto-boot? = false
{1} ok reset-all

SC Alert: Host System has Reset

Sun Fire V210, No Keyboard
Copyright 1998-2003 Sun Microsystems, Inc. All rights reserved.
OpenBoot 4.11.4, 4096 MB memory installed, Serial #xxxxxxxx.
Ethernet address 0:3:ba:xx:xx:xx, Host ID: 83xxxxxx.

{1} ok setenv auto-boot? true
auto-boot? = true
{1} ok boot
.......... lots of output ........
volume management starting.
The system is ready.

xxxxxx console login:

Friday, December 08, 2006

Separate debug symbols

Jeremy Bettis
Mon, 11 Sep 2006 07:20:35 -0700

Someone mentioned that gdb can separate the symbols from the executables, as a motivating factor for removing debug=yes and debug=no. How do you do this? I cannot find a gcc command to write the symbols to a different file, and the support for reading the separate symbol files in gdb seems clunky. Here is what I have found so far.
1) Strip can remove debug symbols, and write them to a separate file:
strip shared_debug_obj/app.exe -o app.exe
strip shared_debug_obj/app.exe --only-keep-debug -o app.dbg
strip shared_debug_obj/lib.dll -o lib.dll
strip shared_debug_obj/lib.dll --only-keep-debug -o lib.dbg

2) gdb can read the separate symbol files, but only if you tell it too:
add-symbol-file app.dbg
add-symbol-file lib.dbg

3) gdb is ok with add-symbol-file on dynamic libraries before they have been loaded. If gdb would automatically look for the symbol files by some algorithm, that would be wonderful. The appeal of being able to ship a stripped version of my software and then being able to drop in the symbol files and start debugging, is almost too good to resist.

Thursday, September 07, 2006

Setting Beetel BX220 (Modem provided with Airtel) in Always On Mode
Though of documenting it "Har baar bhool jata hu"

Configure Machine IP to some in 192.168.1._ subnet as modems default ip is

Default Gateway: (Modem's IP)

Follow following steps:
1 ) Open ADSL's Setup page
2 ) At the left side click on "WAN"
3 ) Remove all rows.
4 ) Add new entry VPI & VCI 1 & 32 respectively.
5 ) In connection type select PPPOE and Encapsulation type is LLC/SNAP - BRIDGING
6 ) PPP username : PPP Password : xxxx
PPPOE Service Name : Airtel Authentication Method : PAP
7 ) Again give next and Save
8 ) After Save the information select Reboot and click Save/Reboot button

After this setup Modem will get a IP from DHCP. We need to assign static IP to PC.
Follow following steps:

1 ) IP: (Modems subnet)
2 ) Subnet:
3 ) Default Gateway: (Modems IP)
4 ) DNS: &

Woof!!! its done. Now you don't need to dial up from your PC.

Let me extend this to capture some configuration settings to NetGears WGR614 WiFi Modem so it connects smoothly to Beetel BX220.

Note: Select is shown with a (x)

1 )
2 ) Goto Basic Settings
a ) Does Your Internet Connection Require A Login?
No (x)
b ) Internet IP Address
Get Dynamically From ISP
Use Static IP Address (x)
IP Address:
IP Subnet Mask:
Gateway IP Address:
c ) Domain Name Server (DNS) Address
Get Automatically From ISP
Use These DNS Servers (x)
Primary DNS:
Secondary DNS:

Default IP of Beetel ADSL modem & Netgear WiFi modem is same and same is the default gateway so lets change Netgears IP address.
3 ) Goto LAN IP Setup
a ) LAN TCP/IP Setup
IP Address:
IP Subnet Mask:
b ) Use Router as DHCP Server (x)
Starting IP Address:
Ending IP Address: