FreeNAS rclone –bwlimit parameter

Currently FreeNAS GUI does not support setting rclone command line options. I needed to limit sync speed, therefore need to modify the –bwlimit parameter. I will show you how you can do that in current version FreeNAS 11.2-U7

Edit the following file

/usr/local/lib/python3.6/site-packages/middlewared/plugins/cloud_sync.py

around line 106 and add bwlimit, example:

        args = [
            "/usr/local/bin/rclone",
            "--config", config.config_path,
            "-v",
            "--stats", "1s",
            "--bwlimit", "25M",

Then restart middlewared:

service middlewared restart

Start the job

Now start your cloud sync job and check that the rclone process is using bwlimit or any other parameter you specified:

ps aux|grep 'rclone'

Note: this will probably only work until you apply the next FreeNAS update.

Install Intel Raid Web Console 2 Utility for XenServer 7

Edit repo file to enable Centos repository:

vi /etc/yum.repos.d/CentOS-Base.repo
baseurl=http://mirror.centos.org/centos/7.2.1511/os/$basearch/
enabled=1

Install Intel raid web console

yum install csh net-snmp net-snmp-agent-libs net-snmp-libs net-snmp-utils
wget https://downloadmirror.intel.com/26267/eng/RWC2_MR6.11.zip
unzip RWC2_MR6.11.zip && cd RWC2_MR6.11/Linux_x64/disk
./install.csh

Add to /etc/sysconfig/iptables just before:
-A RH-Firewall-1-INPUT -j REJECT –reject-with icmp-host-prohibited
COMMIT

-A RH-Firewall-1-INPUT -p udp -m udp --dport 3071 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 5571 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 3071 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 5571 -j ACCEPT

Restart iptables

systemctl stop iptables
systemctl start iptables

Decrease VM disk size in XenServer

In XenCenter go to the VM Storage tab and ‘Add’ new disk.
Set the size smaller than your existing disk.

Ssh into the vm and partition the disk and format as ext4

fdisk /dev/xvdb
mkfs.ext4 /dev/xvdb1
mount /dev/xvdb1 /mnt

Use rsync to copy your filesystem from original disk to new disk.
I also excluded /home/files/* from rsync here but the rest in this list is nessesary to exclude.

rsync -ahPHAXx --delete --exclude={/home/files/*,/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} / /mnt

Remove all old disk mounts(xvda, swap etc) and add the new disk to /mnt/etc/fstab (not /etc/fstab):

/dev/xvdb1 /               ext4    errors=remount-ro 0       1

Install grub into new disk bootloader:

for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done
sudo chroot /mnt
sudo grub-install --recheck /dev/xvdb
sudo update-grub
exit
for i in /mnt/dev/pts /mnt/dev /mnt/proc /mnt/sys /mnt/run; do sudo umount $i; done
umount /dev/xvdb1

Detach disk in Xencenter

Create a new VM in Xencenter, remove the original disk, attach the disk we just made.

Start the VM.

If it starts ok then shut down the old VM and delete it.

Install Intel Raid Web Console 2 Utility for XenServer 6.5

wget ‘ftp://ftp.ntua.gr/pub/linux/centos/5.11/os/x86_64/CentOS/lm_sensors-2.10.7-9.el5.x86_64.rpm’
wget ‘ftp://ftp.ntua.gr/pub/linux/centos/5.11/os/x86_64/CentOS/net-snmp-libs-5.3.2.2-22.el5_10.1.x86_64.rpm’
wget ‘ftp://ftp.ntua.gr/pub/linux/centos/5.11/os/x86_64/CentOS/net-snmp-5.3.2.2-22.el5_10.1.x86_64.rpm’
wget ‘ftp://ftp.ntua.gr/pub/linux/centos/5.11/os/x86_64/CentOS/net-snmp-devel-5.3.2.2-22.el5_10.1.x86_64.rpm’
wget ftp://ftp.ntua.gr/pub/linux/centos/5.11/os/x86_64/CentOS/net-snmp-utils-5.3.2.2-22.el5_10.1.x86_64.rpm
wget ftp://ftp.ntua.gr/pub/linux/centos/5.11/os/x86_64/CentOS/tcsh617-6.17-7.el5.x86_64.rpm
wget ftp://ftp.ntua.gr/pub/linux/centos/5.11/os/x86_64/CentOS/libstdc++-4.1.2-55.el5.x86_64.rpm

rpm -Uvh lm_sensors-2.10.7-9.el5.x86_64.rpm net-snmp-5.3.2.2-22.el5_10.1.x86_64.rpm net-snmp-libs-5.3.2.2-22.el5_10.1.x86_64.rpm net-snmp-utils-5.3.2.2-22.el5_10.1.x86_64.rpmtcsh617-6.17-7.el5.x86_64.rpm libstdc++-4.1.2-55.el5.x86_64.rpm –force

wget https://downloadmirror.intel.com/26267/eng/RWC2_MR6.11.zip
unzip RWC2_MR6.11.zip
tar -zxvf RWC2_linux_x64_installer-16.02.00-04.tar.gz
./install.csh

#add to /etc/sysconfig/iptables before REJECT
-A RH-Firewall-1-INPUT -p udp -m udp –dport 3071 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp –dport 5571 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp –dport 3071 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp –dport 5571 -j ACCEPT

service iptables stop
service iptables start

Install Intel Raid Web Console 2 Utility for XenServer 6.2 SP1

Intel’s instructions are quite complicated and the package contains more files than you need.
Follow these commands, downoad and unpack:

wget 'http://downloadmirror.intel.com/18441/eng/ir3_Linux_x86_RWC2_v13.04.03.01.zip'
unzip ir3_Linux_x86_RWC2_v13.04.03.01.zip 

Check that you have the correct Centos version, should be 5-x:

rpm -qf /etc/redhat-release

Install these requirements before we can launch intel install file:

yum install net-snmp net-snmp-utils --enablerepo=base

Intels .sh file has weird characters in it, lets convert it, chmod it and the run it to install and start services.

tr -d '\15\32' < RunRPM.sh > RunRPM2.sh
chmod 755 RunRPM2.sh 
./RunRPM2.sh

Open up some ports in Xenserver firewall:

lokkit -p 3071:tcp -p 3071:udp -p 5571:tcp -p 5571:udp

Now, you should be able to connect to your server ip from Windows Intel Raid Web Console 2 UI.
At the time of writing that can be found here.

Aside

Mount NFS share as hard in XenServer 6.2

By default XenServer mounts NFS SR with soft option.
That can cause the VM running on NFS to output an IO error and remount its disk as read-only.
The solution is to hard mount the NFS share. This will leave the VM to hang and wait for the share to come back online.

Edit file /opt/xensource/sm/nfs.py and uncommment existing options line under the soft_mount function and add new options line:

#options = "soft,timeo=%d,retrans=%d,%s" % (SOFTMOUNT_TIMEOUT,
# SOFTMOUNT_RETRANS,
# transport)
#options += ',actimeo=0'
options = "rw,tcp,bg,hard,intr,sync,noac,rsize=262144,wsize=262144,timeo=600,retrans=2,_netdev"

More info on this issue:

Intel Modular Multipath driver for Citrix Xenserver 6.2 SP1

I have compiled a new Intel Modular Server multipath driver for Citrix XenServer 6.2 XS62SP1 kernel based on instructions on my earlier post:

Compile an Intel Modular Server multipath driver for XenServer 6.0.2 XS602E005 kernel with XenServer DDK

Download the new driver for XenServer 6.2 XS62SP1 kernel from here:

scsi_dh_alua_intelmodular_cbvtrak_xs62sp1.i386.tar.gz

Looks like this driver rpm is not needed at all since XenServer 6.2.
Only use multipath.conf file from this package.

Debian Wheezy Mail Server – Postfix Dovecot Sasl MySQL PostfixAdmin RoundCube SpamAssassin Clamav Greylist Nginx PHP5

This is a tutorial on how to install an email server on Debian Wheezy 7.

We are going to install the following components:

E-mail server:

  • Postfix
  • Dovecot
  • Sasl library
  • MySQL

Milters:

  • SpamAssassin
  • Clamav
  • Greylist

Webserver:

  • Nginx
  • PHP5
  • phpMyAdmin
  • PostfixAdmin
  • RoundCube

Please replace any text in red with your info.

Here is a scheme about the internals, click on image to get a clear view:

Install E-mail Server - Postfix Dovecot Sasl MySQL PostfixAdmin RoundCube SpamAssassin Clamav Greylist Nginx PHP5 on Debian Wheezy 7 2014

1. Install Debian Wheezy 7

When installing Debian 7 itself, only select ‘SSH server’ when prompted to select software.
If Debian is ready, install the following packages:

apt-get install sudo mc vim

2. Install MySQL server and client

apt-get install mysql-server mysql-client

Enter a MySQL root password when prompted.

Check that mysqld is running:

ps aux

3. Install PHP5 and Nginx

At this point I use the Dotdeb repo to install PHP version 5.5.x

Add Dotdeb repositories to /etc/apt/sources.list file:

deb http://packages.dotdeb.org wheezy-php55 all
deb-src http://packages.dotdeb.org wheezy-php55 all

Add Dotdeb key:

wget http://www.dotdeb.org/dotdeb.gpg
sudo apt-key add dotdeb.gpg

Update apt cache:

apt-get update

Install PHP5 and Nginx:

apt-get install php5-fpm php5-imap php5-mysql php5-mcrypt php5-intl nginx openssl ssl-cert

Make sure php5-fpm and nginx are running:

service php5-fpm start
service nginx start

Create a folders to store web files:

mkdir /home/clients_ssl
mkdir /home/clients_ssl/<subdomain.domain.tld>
mkdir /home/clients_ssl/<subdomain.domain.tld>/logs
mkdir /home/clients_ssl/<subdomain.domain.tld>/tmp
mkdir /home/clients_ssl/<subdomain.domain.tld>/www

Create nginx config for this site:

vi /etc/nginx/sites-available/<subdomain.domain.tld>_ssl

Press ‘i’ and paste the following in vim, replace <yourip> and <subdomain.domain.tld> with your info:

server {

    listen <yourip>:443;
    server_name <subdomain.domain.tld>;

    ssl on;
    ssl_certificate        /etc/nginx/certs/<subdomain.domain.tld>.combined.crt;
    ssl_certificate_key    /etc/nginx/certs/<subdomain.domain.tld>.key;

    root /home/clients_ssl/<subdomain.domain.tld>/www;
    index index.php index.html index.htm;

    location ~ \.php$ {

        fastcgi_pass unix:/etc/php5/fpm/socks/ssl_<subdomain.domain.tld>.sock;
        include fastcgi_params;
        fastcgi_param HTTPS on;

    }

    location ~ /\. {
        deny all;
    }

    access_log /home/clients_ssl/<subdomain.domain.tld>/logs/access.log;
    error_log /home/clients_ssl/<subdomain.domain.tld>/logs/error.log;
    error_page 404 /404.html;

}

Remove the default site and put your site online:

rm /etc/nginx/sites-available/default
ln -s /etc/nginx/sites-available/<subdomain.domain.tld>_ssl /etc/nginx/sites-enabled/<subdomain.domain.tld>_ssl

Create the certs folder.

mkdir /etc/nginx/certs

Put your cerificates in there. Get a valid certificate from a certificate authority or create a self signed certificate.
You can google on how to make one. Make sure to edit your Nginx config file to match the certificate filenames.

Restart Nginx

service nginx restart

Create a php5-fpm config file:

vi /etc/php5/fpm/pool.d/ssl_<subdomain.domain.tld>.conf

Press ‘i’ and paste the following in vim, replace <subdomain.domain.tld> with your info:

[ssl_<subdomain.domain.tld>]

    listen = /etc/php5/fpm/socks/ssl_<subdomain.domain.tld>.sock
    user = u1001
    group = g1001
    listen.owner = www-data
    listen.group = www-data
    listen.mode = 0666
    pm = dynamic
    pm.max_children = 50
    pm.start_servers = 1
    pm.min_spare_servers = 1
    pm.max_spare_servers = 5
    pm.max_requests = 0
    php_admin_value[open_basedir]=/
    php_admin_value[session.save_path]=/home/clients_ssl/<subdomain.domain.tld>/tmp
    php_admin_value[upload_tmp_dir]=/home/clients_ssl/<subdomain.domain.tld>/tmp
    php_admin_value[disable_functions]=dl

Create a user for this virtualhost:

groupadd -g 1001 g1001
useradd --no-create-home -g 1001 -u 1001 u1001

Create socks folder:

mkdir /etc/php5/fpm/socks

Remove the default php pool:

rm /etc/php5/fpm/pool.d/www.conf

Add timezone info to php ini file /etc/php5/fpm/php.ini:

date.timezone = Europe/Tallinn

Restart php5-fpm

service php5-fpm restart

4. Install phpMyAdmin

cd /home/clients_ssl/<subdomain.domain.tld>/www
wget 'http://downloads.sourceforge.net/project/phpmyadmin/phpMyAdmin/4.0.8/phpMyAdmin-4.0.8-english.tar.gz?use_mirror=netcologne'
mv phpMyAdmin-4.0.8-english.tar.gz\?use_mirror=netcologne pma.tar.gz
tar -zxvf pma.tar.gz

Hide pma or bots will try to hack into it:

mv phpMyAdmin-4.0.8-english pma_763773
cd pma_763773
cp config.sample.inc.php config.inc.php

Set the right owner for www and tmp folder:

cd /home/clients_ssl/<subdomain.domain.tld>
chown -R 1001.1001 www tmp

Now you should be able to access pma at: https://<subdomain.domain.tld>/pma_763773/

Now open phpMyAdmin and click on ‘SQL’ on the top menubar. Paste the following SQL queries to create a database and user, replace <password> as you see fit:

CREATE DATABASE postfix;
GRANT ALL PRIVILEGES ON postfix.* TO 'postfix_admin'@'%' IDENTIFIED BY '<dbpassword1>';
GRANT SELECT ON postfix.* TO 'postfix'@'%' IDENTIFIED BY '<dbpassword2>';
FLUSH PRIVILEGES;

5. Install PostfixAdmin

Although you can install it from a standard Debian package I am going to download it directly instead so I can put it under my custom path immediately.

cd /home/clients_ssl/<subdomain.domain.tld>/www
wget 'http://downloads.sourceforge.net/project/postfixadmin/postfixadmin/postfixadmin-2.3.6/postfixadmin-2.3.6.tar.gz?use_mirror=garr'
mv postfixadmin-2.3.6.tar.gz\?use_mirror=garr pfa.tar.gz
tar -zxvf pfa.tar.gz
mv postfixadmin-2.3.6 pfa_746338
chown -R 1001.1001 pfa_746338
cd pfa_746338
sed -i 's/change-this-to-your.domain.tld/<subdomain.domain.tld>/g' config.inc.php

Now edit configuration file config.inc.php and change these values:

$CONF['configured'] = true;
$CONF['postfix_admin_url'] = 'https://<subdomain.domain.tld>/pfa_746338';
$CONF['database_type'] = 'mysqli';
$CONF['database_host'] = 'localhost';
$CONF['database_user'] = 'postfix_admin';
$CONF['database_password'] = '<dbpassword1>';
$CONF['database_name'] = 'postfix';
$CONF['domain_path'] = 'YES';
$CONF['domain_in_mailbox'] = 'NO';
$CONF['fetchmail'] = 'NO';

Go to https://<subdomain.domain.tld>/pfa_746338/setup.php

This setup script should create the nesessary tables to postfix database.

At the bottom of setup.php enter your admin password and click ‘Gererate password hash’.

Edit config.inc.php and add the hash:

$CONF['setup_password'] = '<hash>';

Now enter superadmin account info. You can use this to access PostfixAdmin and configure domains, e-mail accounts, aliases etc.

Try to log in with the admin account here: https://<subdomain.domain.tld>/pfa_746338

6. Install Postfix and Sasl library

apt-get install postfix postfix-mysql libsasl2-modules libsasl2-modules-sql

When prompted, choose ‘Internet Site’.

Use yor domain name as ‘System mail name’: <subdomain.domain.tld>
For example use ‘mail.yourdomain.tld’. Do not use ‘yourdomain.tld’ here if it is going to be one of your virtual mailbox domains.

Create virtual mail user and group:

groupadd -g 3000 vmail
useradd -d /home/vmail -m -u 3000 -g 3000 vmail

Edit /etc/postfix/main.cf:

mydestination = <subdomain.domain.tld>, localhost

and add the following lines:

virtual_uid_maps = static:3000
virtual_gid_maps = static:3000
virtual_mailbox_base = /home/vmail
virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_mailbox_domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf
virtual_alias_maps = mysql:/etc/postfix/mysql_virtual_alias_maps.cf
relay_domains = mysql:/etc/postfix/mysql_relay_domains.cf
virtual_transport = lmtp:unix:private/dovecot-lmtp
smtpd_recipient_restrictions =
 permit_mynetworks,
 permit_sasl_authenticated,
 reject_non_fqdn_hostname,
 reject_non_fqdn_sender,
 reject_non_fqdn_recipient,
 reject_unauth_destination,
 reject_unauth_pipelining,
 reject_invalid_hostname
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
milter_default_action = accept

Create the following files:

/etc/postfix/mysql_virtual_mailbox_domains.cf

hosts = 127.0.0.1
user = postfix
password = <dbpassword2>
dbname = postfix
query = SELECT domain FROM domain WHERE domain='%s' and backupmx = 0 and active = 1

/etc/postfix/mysql_virtual_mailbox_maps.cf

hosts = 127.0.0.1
user = postfix
password = <dbpassword2>
dbname = postfix
query = SELECT maildir FROM mailbox WHERE username='%s' AND active = 1

/etc/postfix/mysql_virtual_alias_maps.cf

hosts = 127.0.0.1
user = postfix
password = <dbpassword2>
dbname = postfix
query = SELECT goto FROM alias WHERE address='%s' AND active = 1

/etc/postfix/mysql_relay_domains.cf

hosts = 127.0.0.1
user = postfix
password = <dbpassword2>
dbname = postfix
query = SELECT domain FROM domain WHERE domain='%s' and backupmx = 1

/etc/postfix/sasl/smtpd.conf

pwcheck_method: auxprop
mech_list: plain login
auxprop_plugin: sql
sql_engine: mysql
sql_hostnames: 127.0.0.1
sql_user: postfix
sql_passwd: <dbpassword2>
sql_database: postfix
sql_select: SELECT password FROM mailbox WHERE username = '%u@%r' AND active = 1

Add postfix user to sasl group:

adduser postfix sasl

Enable secure smtp ports, edit /etc/postfix/master.cf and uncomment:

submission inet n - - - - smtpd 
 -o syslog_name=postfix/submission 
 -o smtpd_tls_security_level=encrypt 
 -o smtpd_sasl_auth_enable=yes 
 -o smtpd_client_restrictions=permit_sasl_authenticated,reject
 -o milter_macro_daemon_name=ORIGINATING
smtps inet n - - - - smtpd
 -o syslog_name=postfix/smtps
 -o smtpd_tls_wrappermode=yes
 -o smtpd_sasl_auth_enable=yes
 -o smtpd_client_restrictions=permit_sasl_authenticated,reject
 -o milter_macro_daemon_name=ORIGINATING

7. Install Dovecot

apt-get install dovecot-imapd dovecot-pop3d dovecot-mysql dovecot-lmtpd

Create file /etc/dovecot/dovecot-mysql.conf.ext:

driver = mysql
connect = host=127.0.0.1 dbname=postfix user=postfix password=<dbpassword2>
default_pass_scheme = MD5-CRYPT
user_query = SELECT '/home/vmail/%d/%n' as home, 3000 AS uid, 3000 AS gid FROM mailbox WHERE username = '%u'
password_query = SELECT password FROM mailbox WHERE username = '%u'

Edit /etc/dovecot/conf.d/10-auth.conf

disable_plaintext_auth = no
auth_mechanisms = plain login
#!include auth-system.conf.ext
!include auth-sql.conf.ext

Edit /etc/dovecot/conf.d/10-mail.conf

mail_location = maildir:/home/vmail/%d/%n:INDEX=/home/vmail/%d/%n/indexes

Edit /etc/dovecot/conf.d/10-ssl.conf

ssl = yes

Edit /etc/dovecot/conf.d/20-imap.conf

mail_max_userip_connections = 10

Edit /etc/dovecot/conf.d/auth-sql.conf.ext

passdb {
 driver = sql
 # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
 args = /etc/dovecot/dovecot-mysql.conf.ext
}
userdb {
 driver = sql
 args = /etc/dovecot/dovecot-mysql.conf.ext
}

Edit /etc/dovecot/conf.d/10-master.conf

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0600
    user = postfix
    group = postfix
  }
}
service auth {
  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
  }
}

Restart services:

service dovecot restart
service postfix restart

You can now add a domain with PostfixAdmin and test your e-mail server.

Any errors are found in logfiles:

/var/log/auth.log
/var/log/mail.log
/var/log/syslog

8. Install Milters

apt-get install clamav-milter clamav-unofficial-sigs milter-greylist spamass-milter

 

clamav-milter:

Update ClamAv database and start the daemon:

freshclam
/etc/init.d/clamav-daemon start

Edit /etc/default/clamav-milter and uncomment the last line:

SOCKET_RWGROUP=postfix

Create a socket folder inside Postfix chroot environment:

mkdir /var/spool/postfix/clamav
chown clamav /var/spool/postfix/clamav

Configure ClamAv milter:

dpkg-reconfigure clamav-milter

Answer questions as follows:

Handle configuration automatically --> yes
User for daemon --> clamav
Additional groups --> none (empty field)
path to socket --> /var/spool/postfix/clamav/clamav-milter.ctl
group owner for the socket --> clamav
permissions (mode) for socket --> 660
remove stale socket --> yes
wait timeout for clamd --> 120
foreground --> no
chroot --> none (empty field)
pid file --> /var/run/clamav/clamav-milter.pid
temporary path --> /tmp
clamd socket --> unix:/var/run/clamav/clamd.ctl
hosts excluded for scanning --> none (empty field)
mail whitelist --> none (empty field)
action for "infected" mail --> reject
action on error --> defer
reason for rejection --> Rejecting harmful e-mail: %v found.
headers -> replace
log file --> /var/log/clamav/clamav-milter.log
disable log file locking --> no
maximum log file size --> 0
log time --> yes
use syslog --> no
log facility (type of syslog message) --> LOG_LOCAL6
verbose logging --> no
log level when infected --> off
log level when no threat --> off
size limit for scanned messages  --> 25

Tell Postfix to use this new milter:

postconf -e 'smtpd_milters = unix:/clamav/clamav-milter.ctl'
postfix reload

 

spamass-milter:

Edit /etc/default/spamass-milter:

Add ‘-m’ so it won’t change the subject header.

Add ‘-r -1’ so Postfix rejects what SpamAssassin flags as spam.

Add ‘-l’ to avoid scanning e-mails sent by logged in users.

OPTIONS="-u spamass-milter -i 127.0.0.1 -m -r -1 -I"

Restart milter:

service spamass-milter restart

Add a dedicated user for SpamAssassin daemon:

adduser --shell /bin/false --home /var/lib/spamassassin --disabled-password --disabled-login --gecos "" spamd

Edit /etc/default/spamassassin:

ENABLED=1
OPTIONS="--create-prefs --max-children 5 --helper-home-dir=/var/lib/spamassassin -u spamd -g spamd"
CRON=1

Update rules and restart the daemon:

sa-update
service spamassassin restart

Tell Postfix to use new milter:

postconf -e 'smtpd_milters = unix:/clamav/clamav-milter.ctl, unix:/spamass/spamass.sock'
postfix reload

 

milter-greylist:

Edit /etc/milter-greylist/greylist.conf:

# For sendmail use the following two lines
#socket "/var/run/milter-greylist/milter-greylist.sock"
#user "smmsp"
# For Postfix uncomment the following two lines and comment out the
# sendmail ones above.
socket "/var/spool/postfix/milter-greylist/milter-greylist.sock" 660
user "greylist"

Edit /etc/default/milter-greylist:

ENABLED=1
SOCKET="/var/spool/postfix/milter-greylist/milter-greylist.sock"

Make a folder for the socket and restart milter:

mkdir /var/spool/postfix/milter-greylist
chmod 2755 /var/spool/postfix/milter-greylist
chown greylist:postfix /var/spool/postfix/milter-greylist
service milter-greylist restart

Tell Postfix to use the new milter:

postconf -e 'milter_connect_macros = i b j _ {daemon_name} {if_name} {client_addr}'
postconf -e 'smtpd_milters = unix:/milter-greylist/milter-greylist.sock, unix:/clamav/clamav-milter.ctl, unix:/spamass/spamass.sock'
postfix reload

9. Install RoundCube

wget 'http://downloads.sourceforge.net/project/roundcubemail/roundcubemail/0.9.4/roundcubemail-0.9.4.tar.gz?use_mirror=heanet'
mv roundcubemail-0.9.4.tar.gz\?use_mirror=heanet roundcubemail-0.9.4.tar.gz
tar -zxvf roundcubemail-0.9.4.tar.gz
mv roundcubemail-0.9.4 rcb_733621
chown -R 1001.1001 rcb_733621

Open phpMyAdmin SQL window and paste:

CREATE DATABASE roundcube;
GRANT ALL PRIVILEGES ON roundcube.* TO roundcube@localhost IDENTIFIED BY '<rcpassword>';
FLUSH PRIVILEGES;

Add initial tables and data:

cd /home/clients_ssl/<subdomain.domain.tld>/www/rcb_733621
mysql -u roundcube -p roundcube < SQL/mysql.initial.sql

Go to https://<subdomain.domain.tld>/rcb_733621/installer/

See that your environment is ok and click Next.

On the ‘Create config’ page, you may want to change the following values:

product_name: <yourproductname>
support_url: <yoururl>
database name: roundcube
database password: <rcpassword>
default_host: localhost
smtp_server: localhost
language: en_US

Click Continue

Change the value in textarea to:

$rcmail_config['use_https'] = true;

Then copy all from textarea and paste the contents to main.inc.php and db.inc.php under /home/clients_ssl/<subdomain.domain.tld>/www/rcb_733621/config

Remove installer folder:

mv installer ../../

Open RoundCube at https://<subdomain.domain.tld>/rcb_733621/

This is it 🙂
If this was of any use please link back to this tutorial.
Thank you.

Sources used to build this tutorial:

https://library.linode.com/email/postfix/postfix2.9.6-dovecot2.0.19-mysql
http://www.bytetouch.com/blog/linux/how-to-linux-mail-server-with-postfix-and-dovecot-on-debian-lenny/
https://lelutin.ca/posts/installing_postfix_-_clamav_-_spamassassin_-_dovecot_-_postfixadmin_on_debian_squeeze/

Remove hanging VDI-s in XenServer CLI

Sometimes a process involving VM vdi can be interrupted and the VDI is left hanging. It cannot be removed in XenCenter. If you select a SR and go under Storage tab you may see virtual disks that have XenServer host as vrtual machine. It usually says “Control domain on host …”. There is a way to manually remove this virtual disk on the CLI console.

Find out the uuid of your hanging vdi:

xe vdi-list name-label=<vm name label>

Now show all volume groups:

vgs

In the response you should identify your SR volume group that the vdi is in. You can do this by looking at the VSize and VFree fields.

Now scan the volume group for vhd-s:

vhd-util scan -m "VHD-*" -f -c -l VG_XenStorage-<volume group id> -p –v

If you see your vhd there then you can now remove it:

lvremove /dev/VG_XenStorage-<volume group id>/VHD-<vdi uuid>

Now re-scan the SR in XenCenter and the hanging virtual disk should be gone.

Move LVMoHBA shared storage repository between different pools or hosts with VM Metadata backup and restore

This can be useful when you have two different pools and you need to move the VM-s from source pool to detination pool.

For example your source production pool has an old XenServer version installed and you do not want to do a rolling pool upgrade.

Instead, you can forget and detach the shared SR from source pool and attach it on the new pool while preserving VM metadata.

To create and move a portable SR using the xsconsole and XenCenter

1. On the source host or pool, in xsconsole, select the Backup, Restore, and Update menu option, select
the Backup Virtual Machine Metadata option, and then select the desired SR.

2. In XenCenter, select the source host or pool and shutdown all running VMs with VDIs on the SR to
be moved.

3. In the tree view select the SR to be moved and select Storage > Detach Storage Repository. The
Detach Storage Repository menu option will not be displayed if there are running VMs with VDIs on
the selected SR. After being detached the SR will be displayed in a grayed-out state.
Warning
Do not complete this step unless you have created a backup VDI in step 1.

4. Select Storage > Forget Storage Repository to remove the SR record from the host or pool.

5. Select the destination host in the tree view and select Storage > New Storage Repository.

6. Create a new SR with the appropriate parameters required to reconnect the existing SR to the destination
host. In the case of moving a SR between pools or hosts within a site the parameters may be
identical to the source pool.

7. Every time a new SR is created the storage is checked to see if it contains an existing SR. If so, an option
is presented allowing re-attachment of the existing SR. If this option is not displayed the parameters
specified during SR creation are not correct.

8. Select Reattach.

9. Select the new SR in the tree view and then select the Storage tab to view the existing VDIs present
on the SR.

10. In xsconsole on the destination host, select the Backup, Restore, and Update menu option, select the
Restore Virtual Machine Metadata option, and select the newly re-attached SR.

11. The VDIs on the selected SR are inspected to find the metadata VDI. Once found, select the metadata
backup you want to use.

12. Select the Only VMs on this SR option to restore the VMs.
Note
Use the All VM Metadata option when moving multiple SRs between hosts or pools, or when using tiered
storage where VMs to be restored have VDIs on multiple SRs. When using this option ensure all required
SRs have been reattached to the destination host prior running the restore.

13. The VMs are restored in the destination pool in a shutdown state and are available for use.

Intel Modular Server multipath driver for Citrix XenServer 6.2 XS62E004 kernel

I have compiled a new Intel Modular Server multipath driver for Citrix XenServer 6.2 XS62E004 kernel based on instructions on my earlier post:

Compile an Intel Modular Server multipath driver for XenServer 6.0.2 XS602E005 kernel with XenServer DDK

This package differs from previous releases, multipath.conf file has been updated based on new research and testing on XenServer 6.2:

http://www.xenlens.com/intel-modular-server-multipath-driver-for-citrix-xenserver-6-2-kernel

Download the new driver for XenServer 6.2 XS62E004 kernel from here:

scsi_dh_alua_intelmodular_cbvtrak_xs62e004.i386.tar

Attach or introduce an existing LVMoHBA Storage Repository in XenServer 6 with Intel Molular Server multipath enabled

Get the SCSI ID:

xe sr-proble type=lvmohba 2>&1

Grab the id between <SCSIid> and </SCSIid>

Get the XenServer assigned device uuid:

xe sr-probe type=lvmohba device-config:device=/dev/disk/by-id/scsi-<your scsi id>

Introduce the existing SR to XenServer:

xe sr-introduce uuid=<your device uuid> shared=true type=lvmohba name-label="NewSR"

Get your host uuid-s:

xe host-list

Create a PBD for each host in the pool:

xe pbd-create sr-uuid=<your device uuid> device-config:device=/dev/disk/by-id/scsi-<your scsi id> host-uuid=<your host uuid>

Switch the connection on for each host:

xe pbd-plug uuid=<uuid from pbd-create command result>

You should now see the SR enabled in XenCenter and you can rescan it under Storage tab to show all existing virtual disks.

The VDI is not available SR_BACKEND_FAILURE_46 in Xenserver 6.1

You may have a situation where you cannot start your VM in another host in the pool and you cannot live migrate that VM to another host in the pool.

This problem can arise when you use ‘/opt/xensource/debug/destroy_domain -domid XX’ on a VM.

Here are some of the errors you may see.

You may get this error in XenCenter:

Internal error: File "xapi_xenops.ml", line 1788, characters 3-9: Assertion failed

Or this error on the CLI:

Error code: SR_BACKEND_FAILURE_46
Error parameters: , The VDI is not available [opterr=VDI 86012149-349d-469d-969d-6ff4ee36595f already attached RW]

The solution is to shut down the VM, forget the VDI-s, rescan the SR and re-attach the VDI-s to the same VM in the same order.

I would suggest doing a full backup of the entire VM before this.

Here is an artice about making a backup of live running VM:

http://www.xenlens.com/export-a-live-running-vm-guest-from-xenserver-host

If you have backed up everyhting you needed, then:

1) Stop the VM in XenCenter
2) In cli: xe vdi-forget
3) In cli: xe sr-scan
4) Reattach the VDI to the VM in XenCenter
5) Start the VM in XenCenter

References:

http://forums.citrix.com/thread.jspa?threadID=323397

http://forums.citrix.com/thread.jspa?threadID=328057

http://www.bl-nk.net/2013/05/sr_backend_failure_46-the-vdi-is-not-available-already-attached-rw

 

Install Seafile on Debian Wheezy 7 64bit with Nginx and MySql

I discovered Seafile in search for an alternative to OwnCloud which could not manage to sync my large collection of files (100Gb and 80000 files). It took many hours just to discover which files to sync. Luckily I found Seafile, which is one of the best open-source file syncing solutions out there. It’s written in Python/Django and C. It’s fast and does exactly what it needs to.

This is a quick tutorial how to set up an entire cloud file server appliance on Debian Wheezy 7 64bit.

 

Install Seafile

Choose your preferred location to store Seafile:

mkdir /home/orgname
cd /home/orgname
wget 'http://seafile.googlecode.com/files/seafile-server_1.8.3_x86-64.tar.gz'
tar -zxvf seafile-server_1.8.3_x86-64.tar.gz
mkdir installed
mv seafile-server_1.8.3_x86-64.tar.gz installed
cd seafile-server_1.8.3

 

Get all Debian packages and dependencies:

apt-get install mysql-server mysql-client python2.7 python-setuptools python-simplejson python-imaging python-mysqldb python-flup nginx openssl

 

Lets install Seafile with MySQL server as backend. Follow instructions and fill in the appropriate info:

./setup-seafile-mysql.sh

 

Let’s create a startip script that controls all Seafile server apps from one place. Save it as ‘/bin/seafile’.

#! /bin/bash

SEAFILE=/home/orgname/seafile-server-1.8.3

ulimit -n 30000

case "$@" in
  start)
    cd $SEAFILE
    ./seafile.sh start
    ./seahub.sh start-fastcgi
  ;;
  stop)
    cd $SEAFILE
    ./seafile.sh stop
    ./seahub.sh stop
  ;;
  restart)
    cd $SEAFILE
    ./seafile.sh stop
    ./seahub.sh stop
    ./seafile.sh start
    ./seahub.sh start-fastcgi
  ;;
  gc)
    cd $SEAFILE
    ./seafile.sh stop
    ./seahub.sh stop
    cd seafile
    export LD_LIBRARY_PATH=./lib:${LD_LIBRARY_PATH}
    ./bin/seafserv-gc -c ../../ccnet -d ../../seafile-data
  ;;
  *)
  echo "Usage: /bin/seafile {start|stop|restart|gc}" >&2
  exit 1
  ;;
esac
chmod 755 /bin/seafile

 

Edit ‘/home/orgname/seahub_settings.py’ and add e-mail server info. In this case we use a smarthost without authentication (ip based access).

EMAIL_USE_TLS = False
EMAIL_HOST = 'your.smarthost.com' # smpt server
EMAIL_HOST_USER = False           # username and domain
EMAIL_HOST_PASSWORD = False       # password
EMAIL_PORT = '25'
DEFAULT_FROM_EMAIL = 'yourname@yourdomain.com'
SERVER_EMAIL = 'yourname@yourdomain.com'

 

Now let’s configure Nginx

Edit /etc/nginx/sites-available/default as follows:

server {

    listen       80;
    server_name  www.yourname.com;
    rewrite ^/(.*) https://$server_name/$1 permanent;	# force redirect http to https

}

server {

    listen 443;
    ssl on;
    ssl_certificate /etc/nginx/certs/cacert.pem;            # path to your cacert.pem
    ssl_certificate_key /etc/nginx/certs/privkey.pem;       # path to your privkey.pem
    server_name www.yourname.com;

    location / {

        fastcgi_pass    127.0.0.1:8000;
        fastcgi_param   SCRIPT_FILENAME     $document_root$fastcgi_script_name;
        fastcgi_param   PATH_INFO           $fastcgi_script_name;

        fastcgi_param   SERVER_PROTOCOL     $server_protocol;
        fastcgi_param   QUERY_STRING        $query_string;
        fastcgi_param   REQUEST_METHOD      $request_method;
        fastcgi_param   CONTENT_TYPE        $content_type;
        fastcgi_param   CONTENT_LENGTH      $content_length;
        fastcgi_param   SERVER_ADDR         $server_addr;
        fastcgi_param   SERVER_PORT         $server_port;
        fastcgi_param   SERVER_NAME         $server_name;
        fastcgi_param   HTTPS               on;
        fastcgi_param   HTTP_SCHEME         https;

        access_log      /var/log/nginx/seahub.access.log;
        error_log       /var/log/nginx/seahub.error.log;

    }

    location /seafhttp {

        rewrite ^/seafhttp(.*)$ $1 break;
        proxy_pass http://127.0.0.1:8082;
        client_max_body_size 0;

    }

    location /media {

        root /home/orgname/seafile-server-1.8.3/seahub;

    }

}

 

Create certificate folder:

mkdir /etc/nginx/certs
cd /etc/nginx/certs

 

Create self-signed certificates or use your own at this point:

openssl genrsa -out privkey.pem 2048
openssl req -new -x509 -key privkey.pem -out cacert.pem -days 3652

 

Nginx is set up, start it:

/etc/init.d/nginx start

 

Make changes to Seafile to support https:

Modify this line in ‘/home/orgname/ccnet/ccnet.conf’:

SERVICE_URL = https://www.yourdomain.com

 

Add this line to ‘/home/orgname/seahub_settings.py’:

HTTP_SERVER_ROOT = 'https://www.yourdomain.com/seafhttp'

 

Start Seafile server services:

seafile start

 

This is it, point your browser to https://www.yourdomain.com

 

Mount another server with sshfs in Xenserver 6

To mount via sshfs we need fuse fuse-libs and fuse-sshfs.

Initiator server is where you want to mount the target server folder.

Target server is where the mountable folder resides.

First, install some packages so XenServer 6 will support mounting with sshfs:

[initiator]# yum install fuse fuse-libs --enablerepo=base
[initiator]# rpm -Uvh 'ftp://ftp.sunet.se/pub/Linux/distributions/fedora/epel/5/i386/fuse-sshfs-2.4-1.el5.i386.rpm'

Now, log in to the target server and set up a reverse tunnel from target to initiator:

[target]# ssh -R 2222:localhost:22 root@foo -p 22

Now you should be in the Xenserver initiator server, create a folder where you want to mount the target:

[initiator]# mkdir /mnt/sshfs

Mount without sshfs compression:

[initiator]# sshfs -p 2222 -o Ciphers=arcfour localhost:/backup /mnt/sshfs

If this is not working and you get an error like ‘read: Connection reset by peer‘ then you can debug by adding an option -d to the sshfs command.

Debian 7.0 Wheezy template for XenServer 6

There is no Debian 7.0 Wheezy template on XenServer 6.1.

Here’s an easy way to clone it from Debian 6.0 Squeeze template:

xe vm-clone uuid=`xe template-list name-label="Debian Squeeze 6.0 (64-bit)" --minimal` new-name-label="Debian Wheezy 7.0 (64-bit)"
xe template-param-set other-config:default_template=true other-config:debian-release=wheezy uuid=`xe template-list name-label="Debian Wheezy 7.0 (64-bit)" --minimal`

Now you can install Debain 7 Wheezy VM-s directly.

Export a live running VM guest from XenServer host

Log in to the host server.

1. List all vm-s and copy the vm uuid you want to backup.

xe vm-list is-control-domain=false

2. Create a snapshot of the vm.

xe vm-snapshot uuid=[uuid of vm] new-name-label=snapshotname

3. Convert the snapshot from a tmplate to a vm.

xe template-param-set is-a-template=false ha-always-run=false uuid=[uuid of snapshot]

4. Save the snapshot to a file.

xe vm-export vm=[uuid of snapshot] filename=filename.xva

5. Delete the snapshot.

xe vm-uninstall uuid=[uuid of snapshot] force=true

6. Now just copy the saved file to a backup server by ssh, ftp, nfs or any other method you prefer.

You can also save the file to nfs share directly in step 4.

Boot a guest VM from CD or DVD in XenServer

In order to boot from cd or dvd you need to change the guest virtualization type from HVM (fully virtualized) to PV (paravirtualized).

xe vm-param-set HVM-boot-policy="BIOS order" uuid=[uuid of your vm]

After you have booted from dvd, change back to fully virtualized mode:

xe vm-param-set HVM-boot-policy="" uuid=[uuid of your vm]

Read more about difference between HVM and PV:

http://serverfault.com/questions/222010/difference-between-xen-pv-xen-kvm-and-hvm