Device enumeration in FreeBSD

Posted: February 17th, 2013 | Author: | Filed under: FreeBSD, storage | Tags: , , , | No Comments »

One of the FreeBSD boxes I was working with came up with ad8 as its disk device. After reading around a bit and talking to a few FreeBSD devs, here is my findings of where did that number “8” come from:

On Intel PCH2 6 Port SATA controller, disk was attached to 3rd port i.e. port #2. (That can also be checked from the BIOS settings.)

Now, ATA stack reserves two device numbers (master/slave) per ATA channel. But, first two channels are reserved for legacy (ISA emulation) mode – PATA. That means, ad0 to ad3 are reserved.

So, if a disk is attached to port #0 master, it comes up as ad4. In this particular case, because the disk was attached to port #2 master, it came up as ad8.


Storage subsystem jargons

Posted: February 4th, 2013 | Author: | Filed under: FreeBSD, storage | Tags: , , , , , | No Comments »

Standards in order of their creation/availability:
SCSI (Serial Computer System Interface), oldest
ATA (IDE) : AT Attachment, later became PATA (Parallel ATA)
SATA (Serial ATA) : better than SCSI
SAS (Serial Attached SCSI) : better than SATA

SCSI/SATA controls RAID.

SAS : costlier, better for critical functions, server applications
SATA: cheaper, used for personal computers

SAS controller can access SATA drives but that’s not true the other way around.

SAS and SATA drives can operate in the same environment while SCSI and ATA cannot. For example, using faster SAS drives for primary storage and offloading older data to cheaper SATA disks in the same subsystem, something that could not be achieved with SCSI and ATA.

CAM (Common Access Method) : a specification for SCSI
CAM provides a formal description of the interfaces in a SCSI subsystem.
Benefits: providing round-robin prioritized transaction queuing, guaranteed transaction ordering even during error recovery, and a straight forward error recovery model that increases system robustness.

CAM is not perfect, a lot of issues related to implementation of the standard.

More to come about CAM and RAID.

Credits:
http://people.freebsd.org/~gibbs/ARTICLE-0001.html
http://en.wikipedia.org/wiki/SCSI
http://en.wikipedia.org/wiki/Serial_ATA
http://www.webopedia.com/DidYouKnow/Computer_Science/2007/sas_sata.asp


Adding battery life info to Spectrwm status bar

Posted: November 18th, 2012 | Author: | Filed under: FreeBSD, productivity | Tags: | No Comments »

I am still a novice user of spectrwm. When my Lenovo T420 died on me because of low battery, I decided to find out how to add battery life info on the status bar.

I borrowed baraction.sh from arch linux spectrwm wiki and modified it to my needs:


SLEEP_SEC=5
LIFE=0
STATUS=0
#loops forever outputting a line every SLEEP_SEC secs
while :; do
LIFE=`sysctl -n hw.acpi.battery.life`
STATUS=`sysctl -n hw.acpi.battery.state`
if [ $STATUS -eq 1 ]
then echo -e " Battery Life: $LIFE% Status: Not Charging" ;
else
echo -e " Battery Life: $LIFE% Status: Charging"; fi
sleep $SLEEP_SEC
done

.spectrwm.conf should contain:

bar_action = baraction.sh

Right now the laptop doesn’t have power adapter connected and my status bar shows: Battery life: 92% Status: Not Charging

Mission accomplished.


Emails: the way I like it

Posted: August 31st, 2012 | Author: | Filed under: fun, productivity | Tags: , , , , , | No Comments »

Sharing my setup of text-based mail client for “efficient” emailing.

It consists of following 3 pieces

fetchmail – fetches incoming messages from SMTP/POP server
procmail – filters messages
msmtp – sends messages
mutt – mail client to view messages

~/fetchmailrc :

set logfile /tmp/fetchmail.log
poll mail.server.com protocol IMAP timeout 60
user "username"
pass "********"
fetchall ssl nokeep norewrite sslcertck sslcertpath /etc/ssl/certs
mda "/usr/local/bin/procmail -f %F -d %T";

~/procmailrc :

#Set on when debugging
VERBOSE=yes

#The directory where your mail folders will be stored.
ORGMAIL=$HOME/mail/
MAILDIR=$ORGMAIL

# The log of procmail actions
LOGFILE=$HOME/.procmail/log

#filtering rules
# any email sent to me@gmail.com will be stored in ~/mail/.hiren/ dir.
:0:
*^(To|Cc|Bcc):.*(me\@gmail\.com)
$MAILDIR/.hiren/

# similarly, you can have multiple rule...

# Catch-all rule for all unmatched email
:0:
$HOME/mail/

For some weird reason procmail wanted me to name the dirs under ~/mail/ starting with a dot i.e. “.”

You can have your mail stored in mbox or maildir format. I like mailbox format as its more flexible. I am not going into details about that.
Please ask “the Internet” for the difference and detailed understanding.

I am using msmtp for sending emails.

~/.msmtprc


defaults
logfile ~/.msmtp.log

account yourmail
host myhost.server.com
from me@gmail.com
#auth on
user me@gmail.com
password *********

account default : yourmail

mutt related settings:

mutt is a very very powerful and customizable tool. I am listing only a few very basic setup settings for muttrc below:


set mbox_type=Maildir
set spoolfile="~/mail/"
set folder="~/mail/"
set mask="!^\\.[^.]"
set postponed="+.Drafts"
set imap_user=username

# The capcability settings
set mailcap_path=~/.mutt/mailcap

# Settings
source ~/.mutt/settings

# Mailbox definitions
source ~/.mutt/mailboxes-local

~/.mutt/settings is another very important file for basic settings:

set hostname="server.com"
set tmpdir="~/.tmp"
set sort=threads
set sort_aux=last-date-received # showing threads with youngest message last
set sort_browser="reverse-date" # showing mbox list default to newest first
set reverse_alias # If there is an alias ... show that.
set quote_regexp="^([ \t]*[|>:%}#])+" # This is default.
set editor = "vim -u ~/.mutt/vimrc"
set abort_nosubject=yes # cancel messages w/o subject: line
set envelope_from=yes
mime_lookup application/octet-stream
alternative_order text/plain text/enriched text/html

set sendmail="/usr/local/bin/msmtp"
set hidden_host # mutt will skip the host name part of $hostname
set bounce_delivered # Postfix users may wish to unset this variable.
set abort_nosubject=no # don't abort compos. when no subj. is set
set quit=yes # just quit please, kthx
set postpone=ask-no
set delete=yes # delete messages marked for deletion w/o confirmation
set mark_old=no # don't change status of new messages to old
set move=no # don't move read messages

# IMAP settings
unset imap_passive
set imap_keepalive=300

# folder format
set folder_format="%2C %t %N %d %f"

~/.mutt/mailbox-local lists your dirs/folders setup for message filtering:

set folder="~/mail/"
set record="~/mail/.hiren"
mailboxes \
= \
=.hiren \
=.kernel \
=.freebsd \
=.random # note: the last line does not end with backslash

Disclaimer: I’ve just shown basic settings here. Let me know if you are looking for any specific info.


memory disk: basic commands

Posted: August 26th, 2012 | Author: | Filed under: FreeBSD | Tags: , , , , | No Comments »

Create a 1G sized file

$ truncate +1G boot.disk

Create memory disk out of the file

$ mdconfig -f boot.disk
md0

$ ls -l /dev/md0
crw-r----- 1 root operator 0, 132 Aug 23 14:57 /dev/md0

Create a partitioning scheme (i.e. gpt below)

$ sudo gpart create -s gpt md0
md0 created

$ sudo gpart show md0
=> 34 2097085 md0 GPT (1.0G)
34 2097085 - free - (1G)

Add a new partition

$ sudo gpart add -t freebsd-ufs md0
md0p1 added
$ sudo gpart show md0
=> 34 2097085 md0 GPT (1.0G)
34 2097085 1 freebsd-ufs (1G)

Create a ufs filesystem

$ sudo newfs /dev/md0p1
/dev/md0p1: 1024.0MB (2097080 sectors) block size 32768, fragment size 4096
using 4 cylinder groups of 256.00MB, 8192 blks, 32768 inodes.
super-block backups (for fsck_ffs -b #) at:
192, 524480, 1048768, 1573056

Mount the filesystem

$ sudo mount /dev/md0p1 /mnt/
$ df -k
/dev/md0p1 1015416 8 934176 0% /mnt

$ sudo mdconfig -l
md0

undo the experiment

unmount /mnt
$ sudo umount /mnt/

detach the disk
$ sudo mdconfig -d -u 0


Freebsd on Freebsd using QEMU

Posted: August 7th, 2012 | Author: | Filed under: FreeBSD, virtualization | Tags: , , | Comments Off on Freebsd on Freebsd using QEMU

My system looks like this:


$ uname -a
FreeBSD xxxx.xxx.net 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64

1) Install qemu from /usr/ports/emulators/qemu

2) Load following 2 modules:

$ sudo kldload aio
$ sudo kldload kqemu

3) Get iso of the guest OS, in my case its FreeBSD so I got one from:
ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/9.1/

4) Create qemu image: (‘man 1 qemu-img’ for more information)

$ qemu-img create -f qcow2 ~/qemu/fbsd9_1beta.img 8G


$ ls -l ~/qemu/
total 652448
-rw-r--r-- 1 hpanchasara hpanchasara 667693056 Aug 7 12:43
FreeBSD-9.1-BETA1-amd64-disc1.iso
-rw-r--r-- 1 hpanchasara hpanchasara 262144 Aug 7 12:44 fbsd9_1beta.img

Then I tried to launch it with: (‘man 1 qemu’ for more information)

$ qemu -boot d -hda ~/qemu/fbsd9_1beta.img -m 512M -cdrom ~/qemu/FreeBSD-9.1-BETA1-amd64-disc1.iso -localtime

But got following errors:

Listing a few to help the web-crawler which intern helps some lost soul like me:
“CPU doesn’t support long mode”
“Can’t work out which disk we are booting from…”
“panic: free: guard1 fail @ …”

Asking “the Internet” helped me narrow down the root causes to following:

1) 32 bit host cannot have 64 bit guest – which was not the case for me.
2) Machine should be virtualization capable – it is.
from /var/run/dmesg.boot, it should have VMX or SVM under “Features”.

FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012
root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
CPU: Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz (3392.37-MHz K8-class CPU)
Origin = "GenuineIntel" Id = 0x206a7 Family = 6 Model = 2a Stepping = 7
Features=0xbfebfbff
Features2=0x179ae3bf
AMD Features=0x28100800
AMD Features2=0x1
TSC: P-state invariant, performance statistics

3) Virtualization support should be enabled in bios – it was enabled.

$ sudo dmidecode | grep VME
VME (Virtual mode extension)

4) Use of incorrect qemu binary – that was the culprit.

“qemu” has different binary versions depending on the platform/systems:

$ qemu
qemu qemu-system-mips qemu-system-ppcemb
qemu-img qemu-system-mips64 qemu-system-sh4
qemu-system-arm qemu-system-mips64el qemu-system-sh4eb
qemu-system-cris qemu-system-mipsel qemu-system-sparc
qemu-system-m68k qemu-system-ppc qemu-system-sparc64
qemu-system-microblaze qemu-system-ppc64 qemu-system-x86_64

Using following I could successfully launch the guest FreeBSD instance:

$ qemu-system-x86_64 -boot d -hda ~/qemu/fbsd9_1beta.img -m 512 -cdrom ~/qemu/FreeBSD-9.1-BETA1-amd64-disc1.iso -localtime

On this first launch, it does FreeBSD install from the specified iso to the .img file. This is a typical FreeBSD installation process.

One the installation is done, you I could launch the VM via:

$ qemu-system-x86_64 -boot c -hda ~/qemu/fbsd9_1beta.img


BSDCan 2012

Posted: June 14th, 2012 | Author: | Filed under: conference, FreeBSD | 2 Comments »

Thanks to FreeBSD foundation for sponsoring my trip to BSDCan2012 This was my first time attending BSDCan and had a lot of fun and learning.

Day:1 05-09-2012 Wednesday
Ports and maintaining/testing them:
– A lot of rant
– how can the dependencies be maintained easily? so that change in one port does not break other.
– Before updating my port how can I know what all would break from the changes?

Lunch: Sandwiches and salad

Vendor summit:
– Who are all vendors/ppl here? What companies are they from? Who do they represent? Why they are here?
– What do you like/do not like in FreeBSD.
– What do you have that you can push to head.
– What is nice to have (features you like).
– what was promised in last vendor summit and by whom? what is the status of that project?

Dinner: Nice Indian food.

Day:2 05-10-2012 Thursday
Virtualization Dev Summit:
– All about FreeBSD virtualization efforts
– Talked a lot about BHyVe, QEMU
– replace taskqueues with ithreads. (whats the difference?)
– backporting BHyVe to lets say 8.0 is _hard_.

Got a nice caffee mocha from “Second Cup”.

Lunch: Sandwiches and Salad

SSH Tutorial By Michael Lucas:
– Usual ssh configuration related things.
– ssh_config, sshd_config.
– Stupid mistakes ppl make.
– Replace passwords with public-keys i.e. pass-phrases.
– Port forwarding / Reverse port-forwarding.

Day:3 05-11-2012 Friday Conference day 1
BSD Multiplicity:
– History of virtualization
– jails/BHyVe
– And general discussion about how/where things are at

IPv6 and FreeBSD:
– offloading problem?
– TEO – TCP Offloading Engine.
– gre(4) – neeed fixing, small enough – Email bjoen.
userland improvements.

tech.slashdot.org/story/12/01/13/2348206/ipv6-only-is-becoming-viable.

freebsd IPv6 todo – a lot is yet to be done.
Bjoern Zeeb – bz@

Fast reboots with kload:
– /sbin/kload
– Look at loader and userboot code – understand how it works.
– userboot.so/kload does image loading/setup but its not the right place according
to the speaker. why?
– bootup code is much different in i386 from amd64
i368 – its the assembly
amd64 – a bunch of c code to setup page-tables
– still experimental but reboots can be really really fast.
– similar thing working fine in linux
http://developers.slashdot.org/story/04/05/05/1251222/reboot-linux-faster-using-kexec

Virtually-Networked FreeBSD Jails:
– VNAT – virtual networking for freebsd, ZFS – god of filesystems and Jails

– Why 3 togather? – creates a _cloud_
ZFS – instant snapshots/clients
Vnet – VLAN/ dedicated network stack for each jail
Jail – for VMs (simple, requires less resources)
– Gotcha:
no pf-ipf – doesnt really matter.
No vlan jails support on rc.d for FreeBSD
– Dtrace with Jails? – can be done?

Day:4 05-12-2012 Saturday Conference day 2
Optimization in ZFS:
by Spectra – Justin Gibbs and Will
– how ZFS works
– copy-on-write
– they worked on some optimizations
– working on a paper which will be published with details
– code will be available (obviously)
– ZFS has many moving pieces
– has a lot of bugs
– needs a lot of testing/test cases
– Lumos – community for ZFS

Amazon Web Services:
by Randy Harper
– EC2
– animoto – amazing example
– No data-center in India

Ethernet Switch Framework:
By: Stefan Bethke stb@freebsd.com
Aleksandr Raybalko ray@freebsd.com
– Adrian initiated it – Qualcomm Atheros HW
– Trimming freebsd to 8 MB?
– configuration mechanism
– need to get actual FLASH file system (not ufs)
with FLASH, things are easy. how?
– Hardware details of the board
– todo: expose IOCTL for userland to do stuff

Recent Advances in IPv6 Security:
– Talked about security issues and concerns in IPv6

Hackers Lounge

This was one of the most fun parts of BSDCan.

– motto: Shut up and code!
– Socialize


Ergo keyboard

Posted: June 14th, 2012 | Author: | Filed under: fun | Tags: , | No Comments »

I’ve never understood why do “non-ergo” keyboards exist in the first place. But that’s just me.

Anyways, I looked at and tried a few ergo keyboards but liked none. So I made my own:

Things you need are a Microsoft Natural Elite Keyboard and a saw.

Ask the Internet for detailed instructions and just make sure you do not cut the keyboard membrane.

Happy typing!


Adding a printer in FreeBSD

Posted: June 7th, 2012 | Author: | Filed under: FreeBSD | Tags: , , | No Comments »

To add a line printer in FreeBSD, you need following:

1) lpd daemon should be enabled and running.

lpd(8) is the print spooler. It accepts printing jobs and sends them to printers defined in /etc/printcap

=> to enable, add following entry in /etc/rc.conf
lpd_enable=”YES”

This entry will start lpd on next reboot.
You can also start it manually:
# lpd

=> make sure its running
# ps awwux | grep lpd
root 1525 0.0 0.0 10072 1620 ?? Is Wed11AM 0:00.00 /usr/sbin/lpd

2) Update /etc/printcap file with printer details:

For example, if printer name is “holi” and it’s sitting at holi.example.com,

lp|holi|holi laser printer:\
:lp=9100@holi.example.com:\
:sd=/var/spool/output/lpd:\
:lf=/var/log/lpd-errs:\
:sh:

You can also setup different filters.

Look up “man printcap” for the details.


Accessing hosted website within the home network

Posted: June 2nd, 2012 | Author: | Filed under: hosting, web | No Comments »

I was having troubles accessing my own hosted website from within my home networks as mentioned in the previous post.

2 things can be done:
1st: Use some free proxy server to access your website (not a very good solution)
or
2nd: Add an entry in /etc/hosts file of the machine that you are using to access the website:
Following is an example from machine I am using to connect to my website, where 192.168.1.5 is the address of the machine that is hosting the website.