It’s So Hard to Say Goodbye to Yesterday

~ Matt Smith ~ 2 Comments

Cake

 
 
 
 

Check out this incredible cake!!  This was made by Roseanne, wife of my friend Mark.  @RedHat — do you need someone to bake for your events?

Motley Crue

As great as it is to be heading off to Red Hat to start a new chapter of my life, it truly is difficult to leave my many friends at UConn. Again, thank you all — for the party, and for 17 years of sharing, mentoring, and friendship as we worked for the greater good of the University.  You will all be missed – please keep in touch!

Cake

[UPDATE 2013-01-18 13:54] The following was just posted to the weekly IT Employee Newsletter:

Matt Smith, a dedicated professional and highly respected employee of UITS announced his resignation from the University effective January 24th. Matt joined UITS as a student employee supporting desktops and distributed servers in 1996. In those 17 years he developed vast knowledge and technical ability. Under his leadership and “can-do” attitude many technical solutions were researched and implemented. Matt’s expertise made him highly sought after over the years by top leaders in the public and private sectors. Matt has always had a strong commitment to UConn; however, opportunity knocked and he decided to leave and accept a job offer with Red Hat, Inc. Matt will be missed by the University community, his friends and colleagues at UITS, and especially by the Linux team he led since 2009. Please join us in wishing Matt and his family all the best.

Thank You!

~ Matt Smith ~ Leave a comment

As I wrap up my time at UConn, I have been greatly touched by the expressions of well-wishes from my colleagues, my friends.  Some I have worked with for only months, some for almost two decades, but I am truly appreciative of the time we have all spent together.

Cake!Today, at my last meeting of the Technology Implementers Group, I was surprised with a beautiful cake (and coffee!!).

Matt and Karen — thank you.  It has truly been a pleasure to work with you all these years.

So long, and Thanks for All the Fish

~ Matt Smith ~ 1 Comment

As I sent out to several mailing lists on 2012-12-20:

Friends,

January 24th will be my last day at the University of Connecticut.  I have been offered, and have accepted, a fantastic opportunity with Red Hat Inc.  Having been in IT at UConn since 1996, this was not an easy decision for me – but this is an opportunity that I just cannot pass up.  I have truly enjoyed working with and learning from you all; I will miss you all greatly.

The Linux/Virtualization team here is establishing a transition plan to maintain service levels.  I will also be working with UITS Leadership to reduce the impact of my leaving on various organizational activities and initiatives.  If you have any concerns or questions, please let me know — I’d like to leave as few “loose-ends” as possible when I leave.

I will remain local, so please do keep in touch.  My personal email address will continue to be [email protected] .

So farewell UConn! Red Hat, here I come!

And for those that can’t use Google: http://en.wikipedia.org/wiki/So_Long,_and_Thanks_for_All_the_Fish

OpenLDAP: Bootstrapping a minimal cn=config

~ Matt Smith ~ Leave a comment

Here is how to bootstrap OpenLDAP’s slapd with an absolutely minimal configuration, without needing an intermediate slapd.conf, with one feature — the local root user (uid=0/gid=0) has “manage” access.  From this point, ldapmodify can be used via ldapi to continue making configuration changes.  This is a good way to start a new server configuration.

Bootstrap:

$> echo 'dn: cn=config
objectClass: olcGlobal
cn: config

dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcAccess: to dn.subtree="cn=config" by dn=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * none
' | slapadd -n0 -F slapd.d

Start slapd:

#> slapd -F slapd.d -h ldapi://foo

Make changes via ldapmodify:

#> ldapmodify -Y EXTERNAL -H ldapi://foo ...

Need a quick proxy via SSH?

~ Matt Smith ~ Leave a comment

Need access to a web site, tucked behind a firewall?  Do you already have a public SSH server set up to allow access, but not sure how to tunnel your HTTP session?  It’s easy!

ssh -N -D 127.0.0.1:8000 <user>@<ssh server fqdn>

That command sets up a SOCKS4/SOCKS5 proxy.  Just configure your Browser/OS settings to use 127.0.0.1:8000 as a SOCKS proxy, and you can now access any site that your SSH server can reach.

July 1 2012 Linux problems? High CPU/Load? Probably caused by the Leap Second!

~ Matt Smith ~ 2 Comments

(Update posted, see below)

As posted in multiple places around the web:

Debian

/etc/init.d/ntp stop
date `date +"%m%d%H%M%C%y.%S"`

Red Hat

/etc/init.d/ntpd stop
date `date +"%m%d%H%M%C%y.%S"`

Update:

This first manifested itself for us in our Java stacks — all of our (dual processor) Tomcat servers were running at a load of 30-40.  However, this is a known (and fixed) kernel bug:

https://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=6b43ae8a619d17c4935c3320d2ef9e92bdeed05d

Apparently, simply forcing a reset of the date is enough to fix the problem:

date -s "`date`"

Build a PaaS using Open Source Software

~ Matt Smith ~ Leave a comment

Discussion about OpenShift.  OpenShift has been fully open-sourced, available on GitHub for local deployment, or directly usable as a hosted solution.

Rule #1: IaaS != PaaS

Virtual machines : Application is not necessarily 1:1

Rule #2: PaaS is not a silver bullet

Great for Self-service deployment of applications, varied volatile workloads (development, testing, scale-up/out), with tightly constrained application rules — which implies standardized deployments from template.

Rule #3: PaaS is about developers — AND OPERATIONS!!!!

Operations becomes about capacity planning, not ticket-drive activities.

Rule #4: Be ready to learn

Developers want languages variety, scaling models, integration models — and they want it automagically

Operations want multi-tenancy, familiar installation, and sane configurations — all reproducible.

What is an application?

Runtime (OpenShift cartridges)

Code (One Git repository per application)

Creating an App

The rhc tools are used to create a namespace (domain), then an application space which includes a name and cartridge type, and push the code.

What do you get from public OpenShift?

A slice of the server, a private Git repository, deployment access.

The PaaS service is comprised of a Broker (director front-end, RESTful) and Nodes.  Each node has multiple “gears” (containers secured with SELinux, constrained with cgroups, and isolated with Kernel namespaces and Bind Mounts).

Extending OpenShift

Custom DNS plugins, auth plugs, security policies, and community cartridges.  Quick-start frameworks can be offered to community too.

LXC and SELinux are the future for isolating and securing OpenShift…

… but right now, there are a many moving parts being used to provide isolation and security.

PaaS demans a new security model

DAC just won’t cut-it, too complicated for PaaS.  MAC (SELinux!) is necessary.

Step 1 – Unlearn this (and embrace SELinux)!

setenforce 0

Step 2 – Learn the ‘Z’ (to see SELinux contexts)

ls -lZ
ps -efZ

(Review of SELinux contexts and syntax provided)

http://fedoraproject.org/wiki/SELinux

Demo – deployment of WordPress to OpenShift, in a VirtualBox LiveCD

The OpenShift QuickStart is available here: https://github.com/openshift/wordpress-example

Migrating Workloads to Red Hat Enterprise Virtualization – a Customer Perspective

~ Matt Smith ~ Leave a comment

Presentation by Qualcomm on their experience migrating from Xen/RHEL5 to KVM/RHEV6.

Straightforward advice — plan, plan, plan, then do.

Qualcomm reduced hardware deployment significantly and simplified management with the RHEV tool suite — significant operational savings.

Qualcomm made extensive use of the virt-v2v tool, but had to modify it (yay Open Source!) to make it cluster (RHCS) aware.  Modifications are shipping with RHEV 6.3.

KVM Technology Review and Roadmap Update

~ Matt Smith ~ Leave a comment
  • KVM is a relatively small piece of code, leveraging Linux for much functionality.  This makes KVM easy to secure and very flexible in meeting future needs.
  • Leveraging Linux means that KVM automatically gains the power of Linux’s hardware support, memory management, network utilities, cgroups, SELinux, etc.
  • Features: RHEL6.3 KVM has all the features of modern hypervisors, without needing 3rd party tools: live snapshots, virtualized disk drivers (VIRTIO), live migration, live block migration, USB passthrough, guest power management, etc.
  • Performance: RHEL 6.3 + KVM holds the top 7 SPECvirt spots on HP and IBM hardware, with metrics showing ~20%+ better performance than VMWare.
  • Single Guest Scalability: Now supports 160 vCPUs and 2TB RAM per guest (with no additional licensing costs!)
  • RHEL 7.0 will include virtual PCI bridges and will have a new Virtio-SCSI block device, enabling thousands of devices per virtual machine.
  • RHEV scales up to 200 host nodes per cluster.
  • Compare the above numbers with VMWare
  • KVM has achieved World Record IOPS: 1,402,720 IOPS on a IBM x3850 X5 for 8KB request using 7 SCSI pass-through devices.. For 1 KB requests, can achieve 1.65M IOPS.
  • RHEV 7 will support Windows power virtualization
  • RHEL 6.3 brings vCPU and memory hotplug to guests
  • KVM has achieved CC-EAL4+ certification with RHEV 5, and is in process of certification with RHEV 6, with sVirt (SELinux wrapped around guests).
  • Decommissioned guest storage can be scrubbed, meeting PCI-DSS standards.
  • Open Virtualization Alliance promotes open source virtualization and KVM ecosystem.

Campground: CloudForms + Splunk

~ Matt Smith ~ Leave a comment

Great co-hosted Red Hat & Splunk discussion about CloudForms-Splunk integration!

Goal: Measure CloudForms utilization by date/time, by user, by cloud povider, and totals.

Simple rsyslog config to send the right data over into Splunk, then just add the “Splunk for Red Hat CloudForms” app — the metrics stated in the above goal are there, right out of the box.  It really is (or at least seems to be) that easy!

And yes — the Splunk guys know Steve Maresca (and UConn) *very* well.