Docker – Running Ubiquiti NVR and Plex

downloadBye bye virtual machines and their inherent OS bloat. Docker and containerization is here…

The trick to containerization is picking the right workload (as with most things). Think about data, its state and where it lives and whether there are any benefits to running as a container.

Both Ubiquiti’s NVR and Plex’s media server software run’s some base application, this app within its own container then maps to data (which can exist outside the instance) that is consistent.

The fun continues when you can update a container (updating the running application), but keeping the data intact at another location. This can really help with version control etc where you can sometimes just point the new container at the data and turn off the old instance. Rollback? easy. Turn off new container and roll back to old.

Of course things are easier if you are running applications that do not change the data.  Both NVR and Plex only index and capture new data (in consistant format), which makes moving between application versions much easier.

The nature of containerization means that the full power of the host is taken into regard. This is different to regular visualization where each guest is limited to the virtual hardware it is assigned. There are of course challenges where resource is congested, but this can also happen in the latter (cpu scheduling, under / over allocation of resources).

Availability also has to be built with containers in mind, with load balances and instances across multiple hosts.

Update : this site has now now been migrated from a VM to 2 x docker containers…. One for MySQL Backend and one for WordPress FrontEnd. Containers can be linked – so the WordPress container can access MySQL container via its own local port. Very cool.

vmware – copy networking port groups to another host

You might be able to do this via “host profiles” but if you do not have the licensing for it, this is the alternative. Very handy if you have over 50 or so port groups. It can be re-run to add to additional hosts as needed.

Install powercli, run the following to obtain your current list of virtual port groups off existing host;

Get-VirtualPortGroup -VirtualSwitch vSwitch0 -VMHost esx-01

Grab the output and place into CSV file or copy straght into Excel…

Following the formatting of the following “esx_switching-generic.csv” import the required data from aboves output. Note : the top line is the headers and should always be at the top of the CSV file.

Type,HostName,SwitchName,NIC,PortGroupName,VLAN,IP,Subnet,KernelGW
Portgroup,esx-02,vSwitch0,,Test_Network1,510,,,
Portgroup,esx-02,vSwitch0,,Test_Network2,511,,,

 

Place this file in a location that is called by this script —

#This script is designed to allow you to configure switches on multiple hosts by
#importing information from a prepopulated .csv file. vMotion switch created based
#on Mike Laverick's posting http://www.rtfm-ed.co.uk/?p=1514
#!!!!!!!Values passed for Type are Case sensitive since comparisons are being made.!!!!!!!

$getinfo = Import-Csv "D:\esx_switching-generic.csv" #need to input locatin of .CSV file

#Connect-VIServer -Server #Need to input appropriate vCenter Server

$getinfo | % {
$Type = $_.Type #!!!! Case Sensitive !!!!!!
$gethost = Get-VMHost -Name $_.HostName
$SwitchName = $_.SwitchName
$PortGroup = $_.PortGroupName
$Nic = $_.NIC
$VLAN = $_.VLAN
$IP = $_.IP
$Subnet = $_.Subnet
$kernelGW = $_.KernelGW

If ($Type -eq "Switch") {
$gethost | New-VirtualSwitch -Name $SwitchName -Nic $Nic
}

#Gets Switch object based on the value for SwitchName (required for several cmd-lets that do not accept Strings)
#'If' statement is used since a vMotion type does not already have a switch configured which will throw up an error.
If ($Type -ne "vMotion") {
$getswitch = Get-VirtualSwitch -VMHost $gethost -Name $SwitchName
}

#Add additional NIC to vSwitch to create a Team
If ($Type -eq "Team"){
$getswitch | Set-VirtualSwitch -Nic $Nic
}

#Add Portgroup to existing switch with VLAN
IF ($Type -eq "Portgroup") {
$getswitch | New-VirtualPortGroup $PortGroup -VLanId $VLAN
}

#Creates vMotion switch and configures vmkernel gateway (located under DNS and Routing in configuration tab)
IF ($Type -eq "vMotion") {

$newvswitch = New-VirtualSwitch -VMHost $gethost -Name $SwitchName -Nic $Nic
$vmotion = New-VirtualPortGroup -VirtualSwitch $newvswitch -Name $PortGroup
New-VMHostNetworkAdapter -VMHost $gethost -PortGroup $PortGroup -VirtualSwitch $newvswitch -IP $IP -SubnetMask $subnet -VMotionEnabled: $true

$vmhostnetwork = get-vmhostnetwork $gethost
set-vmhostnetwork -network $vmhostnetwork -vmkernelgateway $kernelGW
}
}

hyper-v to esx conversion

V2V cold clone process for SBS / exchange / sql / DC’s etc…..    (works in vSphere)

Steps to convert from hyper-v guest to vmware guest

  • Remove hyper-v integrated services while hyper-v guest is running. (if possible)
  • Note down NIC networking details first
  • Clean shutdown hyper-v guest
  • Use StarWind Converter (free) to convert VHD to “dynamically growing VMware” image and use “IDE” as type. Note “SCSI” does NOT work if you have not got the LSI / BUS drivers in the source image, just use the default options – dynamic vmware and IDE.
  • Create VMware guest shell (virtual machine equivalent to hyper-v specs but without the disks) – remove all hard drives, iscsi controller should also disappear.
  • Upload the VMDK to same data store as virtual machine and attach to the virtual guest in the right order. They should be detected as “IDE”
  • Start new vmware guest
  • If you could not remove integrated services (i.e. hot clone), then you MUST disable hyper-v services immediately.
  • Install vmware tools – reboot
  • Remove hidden NIC and other orphaned devices, and reconfigure network card as per original.
  • Windows may need to be re-activated – do this.

Note : converting a machine to use an IDE controller will limit its performance!

OpenSolaris – Headless server

I’ve moved from a CentOS linux distro running vmware server 2.0 to OpenSolaris running VirtualBox. My previous system was totally headless and i wanted something similar to replace it.

I’ve just started getting into OpenSolaris for many reasons (Sun has some cool stuff – ZFS / VirtualBox). But I have always wanted to run OpenSolaris as a headless server, and from what i initially found VirtualBox didn’t have the easy to use autostart on boot features that vmware server had. But there are always ways to get things working…

OpenSolaris 2008.11 has desktop sharing (system->preferences->desktop sharing) which is half of the job — enable this.

I had issues if i used a password protected session (it kept prompting on the actual console for password to unlock the key chain), so chose not to prompt for password for now. Now every time after you have logged in you are able to connect to your machine via VNC. (veno-server)

The other half of the problem is how to have the machine automatically log on as a user on boot-up. This is easily enabled via /etc/x11/gdm/custom.conf or gdmadmin. See this post for more details on the autologon.

Updated : 26/07/2009

After a bit of playing about I’ve found another way to make opensolaris the perfect headless box. First fire up gdmsetup and enable the required remote sessions.

Go to the “Remote” tab and set the style to “same as local”, then under security ensure that “Deny TCP connections to Xserver” is not checked.

Next go into the services GUI and tick the box next to X server (x11/xvnc-inetd). Next type;

svcs | grep vnc

Disable all vnc services except the one we want to enable (below);

svcadm enable xvnc-inetd

Now to get the vnc session to remain open when you disconnect update the service with the following parameter change;

svccfg -s xvnc-inetd setprop inetd/wait = boolean: true

Reboot. Done.

You should upon reboot be able to vnc straight into the box with a session that wont reset on disconnect.

Opensolaris 2008.11 and xvm

I’ve been dabbling with virtual box up until this point. But have decided to take it to the next step by using xvm to run my virtual machines. I have taken a guide from BDerzhavets at this site : http://bderzhavets.blogspot.com/2008/12/xvm-on-opensolaris-2008.html

First step is to open package manager and find the “vitalization” grouping. Install everything in this group. You’ll probably find some packages are already installed.

Next you’ll need to add the xvm kernel to the grub startup menu (menu.1st). This bit is a bit of blind faith from my perspective – its a shame that the xvm kernel package doesn’t add itself to grub? Anyhow… You need to add the following as another boot option in the /rpool/boot/grub/menu.lst file;

title OpenSolaris 2008.11 snv_101b_rc2 X86 xVM
bootfs rpool/ROOT/opensolaris
kernel$ /boot/$ISADIR/xen.gz
module$ /platform/i86xpv/kernel/$ISADIR/unix /platform/i86xpv/kernel/$ISADIR/unix -B $ZFS-BOOTFS

After a bit more reading you can apparently run bootadm -m upgrade to automatically ad the new kernel to the boot menu. But there might be some issues with on-boot? Some good info on this page : http://trevoro.ca/blog/2008/05/07/getting-xvm-to-work-in-opensolaris-200805/

…added the xVM entry automatically after performing the linkage steps. You will still need to add the ‘bootfs rpool/ROOT/opensolaris lines however.

Update : i have not implemented this yet – i’m currently happy with the stable file / virtual box server that i’m using at the moment. Its also a little early to check out this tech, as there is a lot of work being done at http://www.opensolaris.org/os/community/xen/ that is near completion.

VMware Server 2.0 – Infrastructure Client

One thing i like about vmware server 2.0, is that you are not forced to use the web console in all cases.

If you are lucky enough to have access to VMware Infrastructure Client 2.5 you can connect via it using the following syntax at the logon screen;

IP address / Name : http://192.168.9.100

If you enter a URL in the host field, use your root and correct password you should be able to logon without issue. Welcome to a nice fast GUI admin console.

VMWare – Converter Best Practices (p2v)

I found this on the web a while back, it sums up most of what you need to know before doing any P2V work (this generally talks about a windows physical to virtual conversion)…. 
 
 
Converter Best Practices

What should I do after I successfully convert my virtual machine?

If you change from a multi-processor system to a uni-processor system you need to manually change the HAL on the Windows server after the conversion. To do this go into Device Manager after the machine first boots and discovers it’s new hardware and then click on Computer then right-click on the processor and select Update Driver. Then select Install from specific location and then Don’t search I will choose the driver to install. Then select show All compatible hardware and select the appropriate processor. For example, if you went from a dual cpu to a single cpu then select ACPI uni-processor PC instead of ACPI multi-processor PC. You will need to reboot once you change this. To verify what HAL you are using you right-click your hal.dll in c:\windows\system32 and select the Version tab and select Internal Name and it should say halmacpi.dll for multi-processor acpi and halacpi.dll for uni-processor acpi.

Next clean up all the non-present hardware after the P2V conversion. To do this go to a CMD prompt and type;

SET DEVMGR_SHOW_NONPRESENT_DEVICES=1 

DEVMGMT.MSC

then select Show Hidden Devices. Delete any old grayed out hardware. Next remove any vendor specific applications/drivers. For example on a HP server you should go to Add/Remove programs and remove any HP management agents, survey utility, array config utility, version control agent, etc. Also check your NIC and make sure there are no vendor specific drivers there (ie. teaming). Check the Services to see if all there is anything vendor specific related there and disable any services that are.

 

Continue reading VMWare – Converter Best Practices (p2v)

VMWare Performance tips

these are tested from experience…. (the following is true on vmware server 1.0.6 on a linux host)

  • Always use only 1 Virtual CPU – i have found that most times even if you have heaps of cores to spare on the host, a single virtual core is faster.
  • Set the memory to the recommended level when possible – 256Mb sounds shit when you have 4gb, doesn’t matter – set it to 256Mb (increase only if excessive paging on the guest). Let linux and vmware manage the memory, in most cases it does a much better job than windows. Do not disable memory page trimming on any of the guests, this gives more memory back to the host to cache other more important activity (disk etc)
  • Never use 2gb split files for the virtual disk unless you have to transfer the vm onto a file system that doesn’t support larger files.
  • Load the latest drivers for your NIC on the host
  • Be nice to your physical disk – do not load all your VM’s on a single drive – the HDD is the slowest part of the system, dont saturate it. I use either a raid 0 running multiple vm’s (with backups) or dedicated drives per set of vm’s. This excludes enterprise level raids which can handle much higher level of simultaneous requests.
  • Install vmware tools on all guest machines

watch your pings – ping your guests… they should never loose a ping. If they do you are loading the cpu to high and the host cannot process the network queries. As above use only one virtual cpu. Also ensure you have the latest network driver loaded for your NIC on the host.

Virtual Appliance – Virtual Router

This is quite a handy app for creating small virtual routers. See the website for more detail… The commandline is similar to cisco – so its good for practice if you are going for your CCNA.

http://www.vyatta.com/products/virtualized.php

Oh – its open source and free!

I’m a bit lazy in most cases and just add a few virtual nics into my VM’s and do the routing though them. But some people may find this handy for a single point to manage networking.