IVAA Tech Specs

From Transmission
Jump to: navigation, search

Contents

Base System

  • Ubuntu 8.04
  • Packages required: apache2 build-essential python2.4 flvtool2 mencoder ffmpeg2theora python-cheetah python-twisted postfix
  • Users added (with admin privileges) for the tech people
  • machine name is 'video'

Partition Layout

  • 2x500G Hard disks
  • each hard disk partitioned into 1x100M (sdX1) partition, and 1x500G (sdX2) partition. Each of these partitions is set as a RAID volume
  • RAID1 is configured for sda1+sdb1 and sda2+sdb2.
  • md0 (sda1+sdb1) is formatted for ext3, to be mounted as /boot
  • md1 is set as physical volume for RAID
  • One volume group (Sys) is created on md1
  • Three logical volumes are created on Sys: System (10G), Swap (8G) and Data (the remainder, about 480G).
  • System: ext3, mount as /
  • Data: ext3, mount as /opt
  • NOTE: /var/www is a link to /opt/www - all data is stored in /opt

Network Configuration

Media:IVAA_archive.png

/etc/network/interfaces: 

auto eth0
iface eth0 inet static
	address 192.168.4.3
	netmask 255.255.255.0
	gateway 192.168.4.1

/etc/resolv.conf: 

search ivaa
nameserver 192.168.4.3
nameserver 203.130.209.242
nameserver 202.67.222.222

Router Setup - DLink WRT54GL Wireless router

  • WAN IP 192.168.2.250/24 Gateway 192.168.2.1
  • DNS servers 192.168.4.3 203.130.206.250 202.67.222.222
  • LAN IP address 192.168.4.1
  • DHCP range 192.168.4.100 - 192.168.4.255
  • Wireless SSID VIDEO-ARCHIVE, no password (can be added later?)
  • Wireless channel 11

Service Configuration

Bind/DNS

  • /etc/bind/named.conf.local changed to: 
# forward DNS
zone "ivaa" {
   type master;
   file "/etc/bind/zones/ivaa.zone";
   };
  • /etc/bind/named.conf.options changed to (forwarders should be changed to the ISPs DNS servers): 
options {
	directory "/var/cache/bind";

	 forwarders {
		203.130.206.250;
		203.130.209.242;
	 };

	auth-nxdomain no;    # conform to RFC1035
	listen-on-v6 { any; };
};
  • /etc/bind/zones/ivaa.zone changed to (192.168.3.2 should be changed to IP address of server on the network, and the serial number (2009051302) should be changed every time this file is changed): 
$TTL 259200

@           IN       SOA      ns1.ivaa. vik.engagemedia.org. (
                              2009051302     ;serial, todays date + todays serial#
                              14400          ; refresh after 4 hours
                              7200           ; retry every 2 hours
                              2419200        ; expire in 4 weeks
                              86400 )        ; minimum TTL 1 day

      IN NS          ns1.ivaa.    ;Inet address of name server

      IN A           192.168.4.3   ;the static IP of this machine on the network/LAN

ns1   IN A           192.168.3.2   ;same as above
     
; the local server 
plumi-dev    IN CNAME       ns1
flv-dev      IN CNAME       ns1
plumi-prod   IN CNAME       ns1
flv-prod     IN CNAME       ns1
video        IN CNAME       ns1

Zope/Plone

Production Instance
  • Plone 2.5.5 installed via unified installer. If want plone to be installed some other than /opt/Plone-2.5.5 you need to edit install.sh in to change $PLONE_HOME 
wget http://launchpad.net/plone/2.5/2.5.5/+download/Plone-2.5.5-UnifiedInstaller.tgz
tar zxvf Plone-2.5.5-UnifiedInstaller.tgz
sudo mkdir /opt/plone/
sudo mkdir /opt/plone/prod
sudo mv Plone-2.5.5-UnifiedInstaller /opt/
cd /opt/Plone-2.5.5-UnifiedInstaller

Edit install.sh to make $PLONE_HOME=/opt/plone/prod/Plone-2.5.5 

sudo ./install.sh
  • Plumi 0.2.3 installed: 
wget http://andycat.engagemedia.org/plumi-0.2.3-final.tgz
sudo mv plumi-0.2.3-final.tgz /opt/plone/prod
cd /opt/plone/prod
sudo tar xvfz plumi-0.2.3-final.tgz
  • imsvdex installed into plone instance: 
sudo /opt/plone/prod/Plone-2.5.5/Python-2.4.4/bin/easy_install imsvdex
  • added to /opt/plone/prod/Plone-2.5.5/zeocluster/client1/etc/zope.conf and /opt/plone/prod/Plone-2.5.5/zeocluster/client2/etc/zope.conf: 
     products /opt/plone/prod/plumi-0.2.3-final
  • start zope by: 
/opt/plone/prod/Plone-2.5.5/zeocluster/bin/startcluster.sh
  • Plone site with ID 'plumi-prod' was added to zope via the ZMI ( http://localhost:8080/manage from the server. Log in as admin, password as per /opt/plone/prod/Plone-2.5.5/adminPassword.txt)
  • Go to plone site ( http://localhost:9080/plumi-prod),
  • Go to Preferences->Mail Settings. Set mail server to localhost
  • Go to Preferences->Portal Settings to set "Allow users to choose their initial password"
  • Go to Preferences->Add/Remove Products, then install ATVideo and PlumiSkin products.
Development Instance
  • Plone 2.5.5 installed via unified installer. If want plone to be installed some other than /opt/Plone-2.5.5 you need to edit install.sh in to change $PLONE_HOME 
wget http://launchpad.net/plone/2.5/2.5.5/+download/Plone-2.5.5-UnifiedInstaller.tgz
tar zxvf Plone-2.5.5-UnifiedInstaller.tgz
sudo mkdir /opt/plone/
sudo mkdir /opt/plone/dev
sudo mv Plone-2.5.5-UnifiedInstaller /opt/
cd /opt/Plone-2.5.5-UnifiedInstaller

Edit install.sh to make $PLONE_HOME=/opt/plone/dev/Plone-2.5.5 

sudo ./install.sh
  • Plumi 0.2.3 installed: 
wget http://andycat.engagemedia.org/plumi-0.2.3-final.tgz
sudo mv plumi-0.2.3-final.tgz /opt/plone/dev
cd /opt/plone/dev
sudo tar xvfz plumi-0.2.3-final.tgz
  • imsvdex installed into plone instance: 
sudo /opt/plone/dev/Plone-2.5.5/Python-2.4.4/bin/easy_install imsvdex
  • added to /opt/plone/dev/Plone-2.5.5/zeocluster/client1/etc/zope.conf and /opt/plone/dev/Plone-2.5.5/zeocluster/client2/etc/zope.conf: 
     products /opt/plone/dev/plumi-0.2.3-final
  • Zope ports changed:
    • port-base set to 1000 in /opt/plone/dev/Plone-2.5.5/zeocluster/client1/etc/zope.conf and /opt/plone/dev/Plone-2.5.5/zeocluster/client2/etc/zope.conf
    • zodb_db main/server set to localhost:9100 in /opt/plone/dev/Plone-2.5.5/zeocluster/client1/etc/zope.conf and /opt/plone/dev/Plone-2.5.5/zeocluster/client2/etc/zope.conf
    • zeo/address set to 9100 in /opt/plone/dev/Plone-2.5.5/zeocluster/server/etc/zeo.conf
  • start zope by: 
/opt/plone/prod/Plone-2.5.5/zeocluster/bin/startcluster.sh
  • Plone site with ID 'plumi-dev' was added to zope via the ZMI ( http://localhost:9080/manage from the server. Log in as admin, password as per /opt/plone/dev/Plone-2.5.5/adminPassword.txt)
  • Go to plone site ( http://localhost:9080/plumi-dev),
  • Go to Preferences->Mail Settings. Set mail server to localhost
  • Go to Preferences->Portal Settings to set "Allow users to choose their initial password"
  • Go to Preferences->Add/Remove Products, then install ATVideo and PlumiSkin products.

Apache

  • proxy, proxy_http and rewrite mods enabled 
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod rewrite
  • create /etc/apache2/sites-available/ivaa-plumi-prod : 
<VirtualHost *>
    ServerName plumi-prod.ivaa
    ServerAlias plumi-prod video video.ivaa
    ServerAdmin webmaster@localhost 
   
    <Directory />
      Options FollowSymLinks
      AllowOverride None
      Order allow,deny
#      Deny from none
      Allow from all
    </Directory>

<Location />
    Order Allow,Deny
     Deny from none
      Allow from all
  </Location>
   # DocumentRoot /var/www/flv  

    ErrorLog /var/log/apache2/plumi-prod-error.log 

    LogLevel warn

    CustomLog /var/log/apache2/plumi-prod-access.log combined 
    ServerSignature On

    RewriteEngine On

   # RewriteLog "/var/log/apache2/rw.log"
   # RewriteCond {REQUEST_URI} !prod-flv
    # Normalize URLs by removing trainling /'s
    RewriteRule ^/(.*)/$ http://127.0.0.1:8080/VirtualHostBase/http/%{SERVER_NAME}:80/plumi-prod/VirtualHostRoot/$1 [L,P]
    ###                                   ^^^^ Port zope runs on                      ^^^^^ Plone site id
   # RewriteLog "/var/log/apache2/rw.log"
    
   # RewriteCond {REQUEST_URI} !prod-flv
    # Pass all other URLs straight through
    RewriteRule ^/(.*)$ http://127.0.0.1:8080/VirtualHostBase/http/%{SERVER_NAME}:80/plumi-prod/VirtualHostRoot/$1 [L,P]
    ###                                  ^^^^ Port zope runs on                      ^^^^^ Plone site id

</VirtualHost>
  • create /etc/apache2/sites-available/ivaa-plumi-dev : 
<VirtualHost *>
    ServerName plumi-dev.ivaa
    ServerAlias plumi-dev
    ServerAdmin webmaster@localhost 
   
    <Directory />
      Options FollowSymLinks
      AllowOverride None
      Order allow,deny
#      Deny from none
      Allow from all
    </Directory>

<Location />
    Order Allow,Deny
     Deny from none
      Allow from all
  </Location>
   # DocumentRoot /var/www/flv  

    ErrorLog /var/log/apache2/plumi-dev-error.log 

    LogLevel warn

    CustomLog /var/log/apache2/plumi-dev-access.log combined 
    ServerSignature On

    RewriteEngine On

    # Normalize URLs by removing trainling /'s
    RewriteRule ^/(.*)/$ http://127.0.0.1:9080/VirtualHostBase/http/%{SERVER_NAME}:80/plumi-dev/VirtualHostRoot/$1 [L,P]
    ###                                   ^^^^ Port zope runs on                      ^^^^^ Plone site id
    # Pass all other URLs straight through
    RewriteRule ^/(.*)$ http://127.0.0.1:9080/VirtualHostBase/http/%{SERVER_NAME}:80/plumi-dev/VirtualHostRoot/$1 [L,P]
    ###                                  ^^^^ Port zope runs on                      ^^^^^ Plone site id

</VirtualHost>


  • make /var/www/flv/dev-flv and /var/www/flv/prod-flv 
sudo mkdir /var/www/flv/dev-flv
sudo mkdir /var/www/flv/prod-flv
  • create /etc/apache2/sites-available/ivaa-flv-prod : 
<VirtualHost *>
        ServerAdmin webmaster@localhost
        ServerName flv-prod
        ServerAlias flv-prod.ivaa

        DocumentRoot /var/www/flv/prod-flv

        ErrorLog /var/log/apache2/ivaa-prod-flv-error.log 
   LogLevel warn


        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn
        CustomLog /var/log/apache2/ivaa-prod-flv-access.log combined 

</VirtualHost>
  • create /etc/apache2/sites-available/ivaa-flv-dev : 
<VirtualHost *>
	ServerAdmin webmaster@localhost
        ServerName flv-dev
        ServerAlias flv-dev.ivaa

	DocumentRoot /var/www/flv/dev-flv

	ErrorLog /var/log/apache2/ivaa-dev-flv-error.log 
   LogLevel warn


	# Possible values include: debug, info, notice, warn, error, crit,
	# alert, emerg.
	LogLevel warn
	CustomLog /var/log/apache2/ivaa-dev-flv-access.log combined 

</VirtualHost>
  • make sure /etc/apache2/sites-available/default has: 
NameVirtualHost *
  • sites ivaa-plumi-dev and ivaa-flv-dev enabled: 
sudo a2ensite ivaa-plumi-dev
sudo a2ensite ivaa-flv-dev

Indytube

Production Instance

indytube.conf (in /opt/plone/prod/plumi-0.2.3-final/indytube) modified to: 

[encoder]
BE_HOW_NICE=18
CONVERT_THESE=['.avi','.qt','.mov','.mpeg','.mp4','.mpg','.asf','.wmv','.3gp','.ogg','.flv','.m4v','.ogm','.divx']
DO_ENCODING=True
NUMBER_OF_PARALLEL_ENCODERS=1
ENCODER_LOCKFILE_BASE=/opt/plone/prod/plumi-0.2.3-final/indytube/plumi-encoder.lock
POLLTIME=60

[mencoder]
MENCODER_LOCATION=/usr/bin/mencoder

# for older versions of MEncoder.
#MENCODER_OPTIONS=-really-quiet -of lavf -lavfopts i_certify_that_my_video_stream_does_not_use_b_frames -oac mp3lame -lameopts abr:br=56 -ovc lavc -lavcopts vcodec=flv:keyint=50:vbitrate=150:mbd=2:mv0:trell:v4mv:cbp:last_pred=3 -vf scale=320:240 -srate 22050

# for newer versions of MEncoder, we dont need the "lavfopts" line above, plus we add in a higher bitrate, and a sync option
MENCODER_OPTIONS=-really-quiet -of lavf -oac mp3lame -lameopts abr:br=64 -ovc lavc -lavcopts vcodec=flv:keyint=25:vbitrate=260:mbd=2:mv0:trell:v4mv:cbp:last_pred=3 -vf harddup,expand=:::::4/3,scale=320:240  -srate 22050 -ofps 25 -mc 0/10

[ffmpeg2theora]
CORTADO_LOCATION=http://demo.plumi.org/flowplayer/cortado-ovt-debug-0.2.2.jar
FFMPEG2THEORA_COMMAND=ffmpeg2theora

[flvtool2]
FLVTOOL_LOCATION=/usr/bin/flvtool2

[paths]
VIDEO_FILE_DIRECTORY=/opt/plone/prod/Plone-2.5.5/zeocluster/client1/var/files/Members
FLV_FILE_DIRECTORY=/var/www/flv/prod-flv
INCLUDE_FILE_DIRECTORY=/opt/plone/prod/Plone-2.5.5/zeocluster/client1/var/files/Members
INCLUDE_FILE_SUFFIX=.flv.inc
INCLUDE_TEMPLATE=/opt/plone/prod/plumi-0.2.3-final/indytube/plumi.template

[urls]
FLOWPLAYER_LOCATION=http://flv-prod.ivaa/FlowPlayer.swf
VIDEO_SERVER_URL=http://flv-prod.ivaa
SPLASH_IMAGE_BASE=http://plumi-prod.ivaa/Members
SPLASH_IMAGE_FILE=indytube.jpg

[logging]
LOG_FILE=/opt/plone/prod/plumi-0.2.3-final/indytube/plumi-indytube.log
LOG_LEVEL=logging.INFO
  • Flowplayer installed: 
cd /opt/plone/prod/plumi-0.2.3-final/indytube/
sudo unzip flowplayer.zip
sudo cp flowplayer/FlowPlayerClassic.swf /var/www/flv/prod-flv/FlowPlayer.swf


Development Instance

indytube.conf (in /opt/plone/dev/plumi-0.2.3-final/indytube) modified to: 

[encoder]
BE_HOW_NICE=18
CONVERT_THESE=['.avi','.qt','.mov','.mpeg','.mp4','.mpg','.asf','.wmv','.3gp','.ogg','.flv','.m4v','.ogm','.divx']
DO_ENCODING=True
NUMBER_OF_PARALLEL_ENCODERS=1
ENCODER_LOCKFILE_BASE=/opt/plone/dev/plumi-0.2.3-final/indytube/plumi-encoder.lock
###                      ^^^^^^^^^ actual path
POLLTIME=60


[mencoder]
MENCODER_LOCATION=/usr/bin/mencoder
###              be sure this is the right path; svn mencoder may be in /usr/local/bin/

# for older versions of MEncoder.
#MENCODER_OPTIONS=-really-quiet -of lavf -lavfopts i_certify_that_my_video_stream_does_not_use_b_frames -oac mp3lame -lameopts abr:br=56 -ovc lavc -lavcopts vcodec=flv:keyint=50:vbitrate=150:mbd=2:mv0:trell:v4mv:cbp:last_pred=3 -vf scale=320:240 -srate 22050

# for newer versions of MEncoder, we dont need the "lavfopts" line above, plus we add in a higher bitrate, and a sync option
MENCODER_OPTIONS=-really-quiet -of lavf -oac mp3lame -lameopts abr:br=64 -ovc lavc -lavcopts vcodec=flv:keyint=25:vbitrate=260:mbd=2:mv0:trell:v4mv:cbp:last_pred=3 -vf harddup,expand=:::::4/3,scale=320:240  -srate 22050 -ofps 25 -mc 0/10

[ffmpeg2theora]
CORTADO_LOCATION=http://demo.plumi.org/flowplayer/cortado-ovt-debug-0.2.2.jar
FFMPEG2THEORA_COMMAND=ffmpeg2theora

[flvtool2]
FLVTOOL_LOCATION=/usr/bin/flvtool2
### check path of flvtool2

[paths]
VIDEO_FILE_DIRECTORY=/opt/plone/dev/Plone-2.5.5/zeocluster/client1/var/files/Members
### path of 'Members' folder - will be PLONE_BASE/zeocluster/client1/var/files/Members
FLV_FILE_DIRECTORY=/var/www/flv/dev-flv
### path of flv files - this must be the same as where apache serves flv files from.
INCLUDE_FILE_DIRECTORY=/opt/Plone-2.5.5/zeocluster/client1/var/files/Members
### same as VIDEO FILE DIRECTORY ????? CHECK THIS!!!
INCLUDE_FILE_SUFFIX=.flv.inc

INCLUDE_TEMPLATE=/opt/plone/dev/plumi-0.2.3-final/indytube/plumi.template
### Path to plumi.template in the indytube dir

[urls]
FLOWPLAYER_LOCATION=http://ivaa-flv-dev/FlowPlayer.swf
### URL of flowplayer - easiest if put in the root of flv site
VIDEO_SERVER_URL=http://ivaa-flv-dev
### URL of flv web server
SPLASH_IMAGE_BASE=http://ivaa-plumi-dev/Members
### Base URL of plone site ??? - splash images files are served via Plone.
SPLASH_IMAGE_FILE=indytube.jpg

[logging]
LOG_FILE=/opt/plone/dev/plumi-0.2.3-final/indytube/plumi-testing-wetube.log
### path to log file - can keep in indytube dir or /var/log
LOG_LEVEL=logging.INFO
  • Flowplayer installed: 
cd /opt/plone/dev/plumi-0.2.3-final/indytube/
sudo unzip flowplayer.zip
sudo cp flowplayer/FlowPlayerClassic.swf /var/www/flv/dev-flv/FlowPlayer.swf

Init Scripts 

ZOPE_INSTANCES="/opt/plone/prod/Plone-2.5.5 /opt/plone/dev/Plone-2.5.5"
ZOPE_USER=plone

INDY_INSTANCES="/opt/plone/dev/plumi-0.2.3-final/indytube /opt/plone/prod/plumi-0.2.3-final/indytube"

Backups

ZODB Backups

  • backup-zope.py and restore-zope.py added to /opt/plone/scripts. The first line of both scripts (#!/path/to/python changed to point to /opt/plone/prod/dev/Plone-2.5.5/Python-2.4.4/bin/python2.4
  • a script make-ivaa-backup.sh created in /opt/plone/scripts. This contains: 
#!/bin/sh

cd /opt/plone/scripts
./backup-zope.py /opt/plone/prod/Plone-2.5.5/zeocluster/client1 /opt/plone/prod/Plone-2.5.5/zeocluster/server/var /opt/plone/backups/prod
./backup-zope.py /opt/plone/dev/Plone-2.5.5/zeocluster/client1 /opt/plone/dev/Plone-2.5.5/zeocluster/server/var /opt/plone/backups/dev
  • chmod +x /opt/plone/scripts/make-ivaa-backup.sh
  • Created symlink into /etc/cron.daily: 
sudo ln -s /opt/plone/scripts/make-ivaa-backup.sh /etc/cron.daily

Filesystem backups

TODO

Retrieved from "http://wiki.transmission.cc/index.php?title=IVAA_Tech_Specs&oldid=6321"
Personal tools