About Craig, the guy behind RWUCS
25+ years in the industry with a primary focus on System Engineering and Administration.
Navaera Worldwide 05/23 – Present
Team Lead – Site Reliability Engineering
• Administration, architecture and build of Cisco UCS domains for ESX Clusters.
• IBM Blade Center E and H.
• Nimble NAS systems and supporting NAS centric Nexus networks.
• Cisco Catalyst and Nexus administration. Focus on VPC centric architecture.
• VMware administration versions 6 and 8.
• VMware replication for DR fail-over.
American Express 11/2016 – 03/2017
• Architecture and build of Cisco UCS domains to optimally support Openstack Clusters.
• UCS Central integration with pre-existing local UCSM managed, and new UCS domains.
• Upgraded existing UCS Central instances to 2.0(1a) and working with Local to Global Service Profile migration.
• RHEV – Red Hat Enterprise Virtualization Manager administration.
Costco 12/2014 – 11/2016
• Cisco UCS Director (UCSD 5.0) proof of concept build and testing. Automation and Workflows were tested.
• Architecture and build of UCS domains to optimally support ESX Stretched Clusters and VMware vCenter Site Recovery Manager across datacenters.
• Cisco UCS Central expert managing 6200 & 6300 series UCS’s with Global Service Profiles.
• Cisco UCS – Architecture, procurement, build, and daily administration & break/fix of 6248, 6296 and 6300 series FI’s.
• 11/1/2015 got the first 6300 Series UCS in the lab for migration testing from 6200 Series.
• Managing 30 UCS Domains equaling 1,800 servers, 2660 CPU’s, and 23782 cores!
• Managing VCE vBlocks 720-40k.
• Part of team building Costco’s “Warehouse 2.0” virtualized environment including Cisco UCS Mini. This will be the new compute system in Costco stores replacing stand alone HP servers.
• Part of team building Costco’s new international standard compute platforms.
• Managing a 72 C-Series Hadoop UCS Cluster.
Microsoft VidLabs, WA 12/2010 – 12/2014
• Cisco UCS – Architecture, procurement, build, and administration of 6120, 6140, and 6248 series Cisco Unified Computing System’s (UCS) for audio/video encoding farms. UCS build as IaaS. Consulted xbox and other teams at Microsoft Corporation in initial UCS builds. Largest encoding farm architecture running is two UCS Pods (20 Chassis) with a total of 320 M3 Servers and 1PB of Isilon x400 NAS for working and 11PB of nl400 Isilon NAS for mezzanine archive mirrored to another 6PB Isilon system with SyncIQ. Worked with Cisco Advanced Services to create, test, and document UCS Firmware Polices for specialized purposes like video encoding and Hyper-V. Example here: Cisco UCS Bios Policies for Hyper-V
• Cisco NEXUS 7010, 3064 and 3048 switches in a rapidly evolving environment that includes network traffic up to 100Gbps. Utilize vPC, HSRP, Port-Channels and OSPF to ensure fully redundant, high capacity traffic between multiple SAN, NAS and UCS environments located in several different datacenters. Design of collapsed core architecture specialized for video encoding and streaming of content using Cisco UCS and Isilon 10g NAS systems.
• Hyper-V 2012 Cluster integration on UCS project with Cisco Advanced Services. This includes a 64 node cluster with EMC VNX SAN for Hyper-V CSV.
• Coordinated Cisco Advanced Services and EMC/Isilon in best practice UCS “Pod” architecture for Audio/Video encoding. Believe this is a better 10g UCS NAS architecture than a NetApp vBlock.
• EMC VNX 5300-5600 series SAN devices for MS-SQL, Hyper-V, and UCS boot LUN’s.
• UCS, Isilon, and Cisco network architecture with 1PB X400 series 12 node cluster encoding tier and 6PB Isilon NL series cluster of 48 nodes mirrored with SyncIQ to another 48 node cluster for mezzanine storage and backup. The x400 are also used for Hyper-V iSCSI target volumes.
• UCS/X-IO 2012 Hyper-V build with Cisco Advanced Services onsite for best practices and performance tuning and benchmarking. Setup for Windows and Ubuntu servers.
• Administration of complex ACLs and Cisco ASA 5585 and 5540 firewalls to maintain high-security standards endorsed by the Motion Picture Association of America to ensure digital intellectual property is protected.
• ACL requirement capture, creation and ongoing administration between VLAN interfaces on Cisco Nexus switches. This includes dual vPC with Nexus 3k and 5k pairs connected to UCS to ACL inter-VLAN communication.
• SAN Engineer in a multi Petabyte 200+ server environment with Brocade fabric.
• StorNext filesystems, HSM, and tape library management (300+ Nodes on shared SAN filesystem). StorNext DLS/DLC setup and performance testing.
thePlatform, WA 1/2009 – 9/2010
Senior Systems Engineer
• HP EVA 4000/6000 SAN device build and administration.
• NetApp Filer v3140 and v3016 including backend to VMWare ESX clusters.
• Sun Storage 7310 NAS Devices.
• HP Blade c7000 enclosures SAN and NAS attached. Including Virtual Connect and Flex10.
• Cisco Unified Computing System (UCS) 5108 running ESX cluster.
• Implementation and administration of GSA (Google Search Appliance).
• VMWare Virtual Center, ESXI Server, VMWare Server, and Windows and Linux template build. These are attached to SAN and NetApp Filers. Performance analysis in this environment.
• SAN and NAS attached VMWare server build and administration including performance monitoring and tuning.
• Oracle 11GR2 with ASM architecture and build as virtual and physical clusters. VM clusters were built on VMWare ESX servers. Both SAN attached and VM’s shared virtual disks on NFS mounts.
• MS-SQL and MySQL cluster build and administration.
• New hardware and software review, purchasing, and VAR relationships.
• Capacity planning and report presentation for new hardware purchasing.
• Linux performance lead and build standards lead.
• Interviewer and part of decision making process of new candidates.
• Deploy, configure, and maintain Brocade SAN switches in multiple switch dual fabric environment.
• Daily build and administration of RedHat Enterprize, CentOS, and Windows 2003 Server.
• Monitoring setup with HP SIM, HP Performance monitor for EVA SAN devices, SNMP, Cacti (including uploading new device templates to Cacti project), Zenoss, Munin, Ganglia. Perl programming for Ganglia monitors.
• Manage Active Directory. Manage DNS on Windows 2003 server and Linux.
• New requirement based builds in QA/Lab environment and validation testing for production environment.
• Manage Agile (through Rally) backlog and user stories for System Engineer group.
Visto, Seattle (Now “Good Technology”), WA 6/2007 – 10/2008
Senior Systems Engineer
• Analyze and initiates configurations and other changes within the system per requirements.
• Balance requirements with technical feasibility and sets expectations on new projects.
• Interface with vendors as needed to maintain optimal operations.
• Hitachi SAN build out and administration models 9570 and AMS200 & AMS500.
• EMC SAN administration model Clariion AX4.
• Brocade SAN switch administration and architecture.
• Veritas 4.1 and 5.0 VCS, SFRAC build, upgrade, and administration.
• Veritas cluster file system (CFS) 3.5-5.0.
• RedHat 4 & 5 ES and Oracle Linux clustering with Oracle RAC native clustering.
• Oracle ASM integration with Linux native multipathing (device mapper and multipathd) in SAN environment.
• Solaris and Linux performance tuning.
• Linux system builds with SystemImager and CloneZilla.
• Participation in on-call shift and emergency maintenance coverage.
• Solaris system buildes with Jumpstart.
• System monitoring and proactive response with Cricket, Ganglia, and Nagios.
Global Technology Resources Inc, Denver, CO 11/2006 – 1/2007
SAN Designer (contract)
• SAN implementation using SUN 6130 Arrays, SUN 5230 NAS gateways, and Brocade switches.
• Solaris 10 administration in the SAN environment.
• SUN tape libraries attached to SUN Solaris 10 servers under Veritas management.
• SUN Blade servers attached to HA SUN Solaris clusters.
• Documentation of build out and testing of complete environments sent to datacenter’s for installation.
• Performed basic Exchange Administration on Windows 2003 Exchange Server.
IBM, Boulder, CO 2/2002 – 1/2006
• Architecture, implementation, and steady state of enterprise size SAN.
• SAN architecture over 200 TB.
• NAS architecture over 60 TB.
• AIX, Linux, and Solaris Systems Administration.
• Performed Aix, Linux, and Solaris server builds.
• Identified performance issues on the Unix platform.
• Daily configuration changes to McData and Brocade switches and ESS Shark systems.
• Documentation and creation of process.
• Creation of data gathering SAN agents that centralize topology in SQL database.
• Working with Tivoli TSM (3.7-5.1).
• Tivoli TDP for Oracle, DB2, and Notes.
• Veritas technical team lead and Veritas NetBackup administration.
• FastT Storage Devices.
• Development in LAMP environment.
• Created automated agents in Perl that collected SAN configuration (switches & LUN Masking) and propagated MySQL database with Web Frontend (MDM). Also created Perl programs to constantly audit TSM servers and attached tape libraries alerting on full storage pools and down tape drives. Integration into IBM ticketing system was part of development on both projects. Both are still currently used my IBM internationally.
Capacity Technologies Inc., Santa Monica, CA 6/2000 – 1/2002
Senior Network Engineer
• Creation of research papers and presentations presented to Venture Capitalists on bandwidth utilization focusing on bursty environments, time discipline, current encryption, and VPN technology.
• VMware system management in production environment.
• Use cases, and requirement capture.
• Extensive UML diagram creation consisting of, sequence diagrams, and class diagrams in Rational Rose, and DOME.
• Protocol development.
• Network simulation with Opnet (www.opnet.com) modeler for QoS and traffic analysis over DSL and HFC (cable) networks. This includes C/C++ and Perl work to create custom objects and parse results from Opnet reports.
• In depth research of the DOCSIS protocol including FDM and TDM breakdown to study upstream and downstream performance issues.
• VPN, Firewall, and IDS engineering using Cisco equipment.
• UNIX administration (SUN/Solaris 2.8, Linux RedHat 6.2/7.0, Debian) including following applications Apache/Jakarta, DNS (BIND), Sendmail, NFS, Samba, Atalk, NIS, NIS+, and sshd
StorageTek, Pasadena, CA 1/1998 – 7/2000
Managed Storage Administrator
• SAN Architecture and Administration.
• Firewall planning and implementation.
• Y2K remediation of Solaris 2.5.1-2.6 and Veritas 3.0-3.2.
• Migration of Solaris servers, and Tape Libraries including corresponding Veritas HSM databases.
Visual Perspectives Internet, Pasadena, CA 4/1997 – 12/1998
Senior Network Engineer
• Project management overseeing the full life-cycle of software projects, from the recognition of a need, through requirement capture, specification and planning, prototyping, implementation, testing, and routine use.
• Extensive experience interfacing with top management, accommodating strategic requirements, making briefings and presentations.
• Cisco router and Switch administration.
• Unix system administration consisting of Sendmail, Procmail, Apache Web Servers, DNS (directory name services), kernel configuration, FTP servers, POP3, security, back-up management, proxy servers, and firewall systems.
• Frame Relay set up and administration including T1 provisioning, Cisco and Ascend routing, private virtual circuits, and point to point set ups with custom multiplexing.
Earthlink Network Inc., Pasadena, CA 10/1996 – 4/1997
Frame Relay Specialist
• Installation of Frame Relay connections from clients to Earthlink Network.
• Configuration of many brands of routers for Frame Relay customers.
• Cascade Frame Relay Switch, Cisco, Sendmail, and DNS (directory name service) administration for Frame Relay clients.
Kinko’s World Headquarters, Pasadena, CA 2/1995 – 8/1996
Computer Network Administrator
• Novell Network Administrator at Kinko’s World Headquarters.
GreenSoft Corporation, Pasadena, CA 11/1994 – 1/1995
• Junior Visual C++ programmer.
Naval Construction Battalion 40 7/1989 – 7/1994
• Construction Mechanic E-4
EDUCATION & TRAINING
• Tactical First Aid (Feb 2016)
• Implementing Cisco IP Switched Networks (SWITCH V2.0) (Nov 03 2014 to Nov Nov 07 2014)
• Isilon Advanced Training Class (10/27/2014 – 10/30/2014)
• Implementing Cisco IP Routing (CCNP ROUTE v2.0) (Aug 25 2014 to Aug 29 2014)
• Implementing Cisco Data Center Unified Fabric 5.0 642-997 DCUFI training (Jul 14 2014 – Jul 18 2014)
• MR-1CP-VNXUNIDM VNX Unified Storage Management (02/03/2014 to 02/07/2014)
• MR-1CP-ISIAM Isilon Administration and Management (12/09/13 to 12/13/13)
• Cisco Data Center Support for UC Specialist (Certified 11/29/11)
• Cisco Unified Computing Technology Support Specialist (Certified 11/29/11)
• CCNA (Cisco ID: CSCO12060339) (Certified 5/21/12)
• Quantum StorNext training from vendor.
• Brocade Switch and SAN Manager Product Training.
• Tivoli 5.1 training (IBM).
• Understanding Hardware, AIX Internals, and Performance (based on RS/6000 architecture) (IBM).
• Requirements Capture (Odegard Labs, Inc. http://www.odegard.com 6/2000).
• Software Inspection (Odegard Labs, Inc. http://www.odegard.com 6/2000).
• Evolutionary Delivery (Odegard Labs, Inc. http://www.odegard.com 6/2000).
• Earthlink job training: TCP/IP, Cisco Routers, and Frame Relay.
• Novell Certified Network Administrator 1/95.