Old school troubleshooting based on OSI model

Intro

In college, the OSI model was drilled into me, as one of the IT admin courses I was taking included the full Cisco Certified Network Admin track. While I never did take up a job doing pure network stuff, what I learned about the OSI model is still useful to this day

We have paid tools such as the excellent ControlUP / eG Innovations to aid us with troubleshooting virtualized environmental issues , it’s always good to triage your performance / outage with some good ‘ol fashioned troubleshooting that doesn’t need to make use of paid tools. If you’re a consultant like me, you aren’t always going to be able to install/configure paid tools for your clients, or perhaps the client is VERY small, 1 or 2 virtualization hosts, so , it doesn’t make sense to setup these tools either.

This blog post will review the OSI model, and provided relevant examples for each layer go over the basics, with examples that are applicable to virtualized environments

What is the OSI model?

Let’s start with the below image from imperva

As above, I learned of the OSI model in college, so, I had test/exam/quiz questions on it, to remember the 7 layers of the cake, there are a few acronyms you can use. I chose the following

ALL
PEOPLE
SEEM
TO
NEED
DATA
PROCESSING

I’ve been working in IT professionally (whatever that means to you) for 21 years. I believe that’s a pretty good chunk of time to have formed an efficient method for troubleshooting networked computers. From experience, the most complicated (and therefore lengthy ) issues you DON’T want to start your troubleshooting steps with are layers 4-7. The worst, is application troubleshooting. Why? Way too many variables at play: settings, interoperability with the underlying operating system / drivers/ runtimes. Trust me, you don’t want to start you path here. Instead, I always start with the ground floor of your network device /computer, the physical layer

1 – Physical layer

Here, you can review the basics at the physical layer that might be the culprit for your performance issues. Our first bullet list will mostly cover those physical / hardware settings that would be relevant to check on your virtualization hypervisor

  • What is the expected link speed of the network card? I had an HP desktop unit I’d re-purposed for use with ESXi last year, I was noting issues with the mgmt. interface going down, the built-in NIC was supposed to run at 1 Gbe, but refused to go beyond 10mbit/sec, yikes! I ended up disabling it, and cutting over to a dual-head 1GB intel NIC
  • Modern SSDs using the NVM protocol and connect using M.2 or U.2 on the motherboard. However, there are still plenty of regular SATA-based drives that connect using traditional SATA cables, of which there are TWO cable types. 3 Gbit/ 6Gbit /sec, check that you’ve got the right cable connected to ensure full bandwidth
  • Not all PCI express slots are created equally. In some systems, you’ll only get one of your x16 PCI express lane running at full speed. So, if you have a fancy 10 Gbe NIC in your virtualization hosts like me, but also have dual or quad head 1 Gbit NICs, place the slower NICs in the down-shifted X4 PCI slot, and the 10 Gbe nic in the x16
  • RAM: Ensure your virtualization host isn’t mixing / matching RAM speeds, else, you get the slowest speeds
  • ESXi by default will install and set itself to “Balanced” power mode, this means your CPU will max out at the 80% mark, unless you’re working in a “green” or LEED certified datacenter, you’ll want to set it “MAXIMUM PERFORMANCE” as such:

2 – Data Link

A few simple items to check at the data link layer include the following:

  • VLANs set on your network
  • A manually coded MAC address set on your network card, it’s possible you or someone else set one as part of a PVS implementation
  • The data link layer is also where error control is handled, so, if you’re using PVS, you’ll want to inspect your network switches for any corresponding error messages

3 – Network

As we move up the layers , our troubleshooting steps will be become more complex and involved, this is where you’ll more than likely need to start using built-in CLI tools for your OS: *nix or windows, here’s a good starting list, I’ve ordered them based on what to start with:

Ping from a regular command line or test-netconnection via powershell, you can also test individual ports via test-netconnection -computername XXYY -port 123

-Netstat -ano: to show a mass list of port bindings to IP addresses , very handy if your troubleshooting issues with service ports on on-prem CVAD controllers where you suspect a binding isn’t working as expected

-Tracert: I’ve used this countless times to ID issues with default gateways, that is, where a virtual device isn’t getting the correct default gateway. You’ll see this in a tracert, where the routing process will fail after hop 1 or 2, in a large enterprise, you could have 5-6 hops to get all the way from source to destination

-Nslookup: useful for testing basic name resolution , Citrix / VMware / Nutanix require static DNS entries for some functionality , if you suspect an entry is incorrect / missing, and need to update the DNS record in MS active directory DNS or unix BIND, start with an nslookup first

-WireShark: This tool is extremely powerful, it’s actually useful for troubleshooting problems from layers 2 all the way to 7! However, it can certainly be used for the NETWORK LAYER to start

4 – Transport

Here is where TCP/IP based communication lives. There are some quick wins to be had here in terms of troubleshooting performance . The first relates to a cursed feature set that Microsoft developed for use with Windows Server 2003, and it’s been a thorn in many an IT person’s side ever since! It’s called Scalable Networking Pack – Wikipedia, the Microsoft article is here

For years, blogs/tech/articles/or even MS support engineers would recommend disabling RSS, receive side scaling. When modern PC’s only had a single vCPU or pCPU assigned, this made sense to have it disabled, but when was the last time you worked on a system with a single core? However, the RSS setting you might have put in place as best practice years ago via script/GPO/GPP/BAT/3rd party tool doesn’t actually get reset when you update VMware tools, as such, you should evaluate ALL of the following settings on the network card properties of your physical/virtual computer. Each should be tested and enabled where possible to ensure optimal performance

4k jumbo frames setting: Here you will want to use a proper method to ID the maximum MTU window size you can set on your physical network cards, switches and virtual network cards. You can do this via VMKPING on ESXi, and ping -L on windows

TCP chimney offload

Receive side scaling

5 – Session

To be honest, you will probably never do any pure troubleshooting at the session layer. Instead, you will troubleshoot layer 7 application issues that can cause Citrix “session” disconnects due to presentation layer (TLS security) or network issues that can occur along the way through layers 1-4. The session layer would be considered a “legacy” layer as we transition to connectionless based protocols such as UDP. However, TCP is a valid fall back for UDP sessions, so, it’s important to know the basics of session based troubleshooting. For troubleshooting Citrix-based environments using receiver / workspace app, refer to the previous layer troubleshooting notes and read on for the final two layers!

6 – Presentation layer

This is where encryption related issues can occur. All the tech vendors will remove support for older versions of TLS as time goes on, ensure that you’ve got the right TLS versions configured on your networking devices and end-points to align with industry best practices. Protip: A certificate that’s expired on your ADC might not initially appear as a “your cert has expired” on workspace app , use a daily health check, or set a calendar reminder when your ADC / StoreFront certs are coming up on renewal

7 – Application

As I mentioned in the first part of this blog post, this is the least fun / most complicated layer to troubleshoot, hopefully, the troubleshooting you’ve done with layers 1-6 have removed the need to keep the party going @ layer 7. If not, let’s get into it!

Windows updates:
As per Microsoft’s insane release schedule, windows is never really the same for long. Monthly patches will fix some issues, close security holes, and open new security holes and create new issues. Identifying if the most recent windows update patch you applied comes down to good testing. If you’re running a virtualized environment , silo off a section of VMs to deploy the latest windows updates, and provide them to dedicate test users, as well, ensure you’ve got a formal testing process that can be filled out and tracked each month

Runtimes:
Vendors such as Citrix/VMware/Nutanix/etc will include updates to pre-req programing language runtimes required with their apps, but they often don’t install the latest versions. To determine if you’re most recently installed vendor binary hasn’t created a new issue with pre-existing installed runtimes, reboot the machine, and open eventviewer and check for any new administrative events related to missing/replaced/corrupted runtimes.

Vendor updates:
There are a few schools of thought on how best to deploy vendor updates: One is evergreen, where you are always on the latest version of the application. The other is follow a semi-annual , LTSR or quarterly approach. I’ve found the best approach, is a hybrid based on what makes sense, and will reduce the impact should something go wrong, or you have an environment where proper UAT is not possible, and new environment changes can only be validated in production

Here’s what I use for most of my clients that are on a windows 201x / Win 10 based virtual desktop:

To cover the need to patch for critical security vulnerabilities, the following elements can be patched monthly

  • The windows OS
  • Stick with an patch the “current release” channel of MS office

Also to cover the need to patch for security issues, you’ll want to review the release notes on Google Chrome / Edge Chromium and patch monthly. That being said, both vendors are tightening the noose with legacy browser settings, you may find your clients internal web apps stop working as expected if you update to the latest version of either of the above browsers, always test, or if this is not possible, send out a comm to the IT/business contact with a link to the release note

Middle-ware apps such as WEM/FSLogix/CVAD/Citrix Optimizer should be updated as required when there are features you want, I’ve had to back out of plenty of changes for multiple clients where I blindly installed the latest version of each . FSLogix QA has certainly suffered since the MS takeover a few years back, the last thing you want is to introduce a prod issue with your profile management solution “just to be on the latest” version. Again, unless there’s a critical feature / fix you want in your middleware app, and you’re still supported should you need to escalate to the vendor, leave it alone

General tips / Wrap-up

The meat of the above blog post advocates for troubleshooting based on a basic understanding of the OSI model, that being said, it’s not always practical to through all the layers, from experience, a huge chunk of the time we spend on “Why is Citrix slow” comes down to the application level. The ace in the hole for me in troubleshooting application / system issues in virtualized Citrix environment, is basic Powershell loops, you can make these more advanced, and I certainly recommend my own github, or Sasha Tomet’s for daily environment health checks if you want to go more advanced, but let’s start with a basic example:

User’s are intermittently complaining they can’t connect to a non-persistent Citrix VDI desktop

Obviously, your first steps should be review Citrix studio / director for anything obvious, however, this route may not provide you with root cause on your issue

One of my favorite uses for Powershell, is to mass collect event IDs, the below code reads all VDs from a delivery group, filters out offline assets, and checks for specific win event IDs, you can edit the event ID to suit your own needs

$ReportTime = (Get-Date).ToString('MM-dd-yyyy-hhmm-tt')
$ReportsPath = "\\UNC\PATH\TOFOLDERTOSAVEXLS"

Add-PSSnapin Citrix*

IF (Get-PSsnapin -name  Citrix.Sdk.Proxy.V1) {

    Get-XDAuthentication -CustomerId CtxCloudID

}

Else {

    Write-warning "Please install Citrix Powershell SDK to access Citrix Cloud, path below"
    write-warning "The script will now exit"
    EXIT
}

If (-not(Get-Module -ListAvailable ImportExcel)) {

    write-host "Installing ImportCSV module and pre-reqs"
    Install-Module ImportExcel

}


### CTX Cloud Win 10 VDI Prod Leo/Molson
write-host "Collecting asset info from Citrix cloud" -ForegroundColor Cyan

$Assets = Get-BrokerMachine -DesktopGroupName "Your DG NAME HERE" -MaxRecordCount 2000 | Where {$_.PowerState -eq "On"} | Select MachineName, @{E={$_.AssociatedUserFullNames};Label='AssociatedUserFullNames'}, @{E={$_.AssociatedUserNames};Label='AssociatedUserNames'} , SessionClientName, SessionClientVersion, SessionStartTime, DesktopGroupName

## #Start!

$AssetsTotal = $Assets | Measure | Select-Object -ExpandProperty Count
$AssetsLeft = $AssetsTotal

$OutArray = @()

ForEach ($i in $Assets) {

    $VM = $i.MachineName.Split("\")[1]
    $User = $i.AssociatedUserFullNames
    $AssociatedUserFullNames = $i.AssociatedUserFullNames
    $AssociatedUserNames = $i.AssociatedUserNames
    $SessionClientName = $i.SessionClientName
    $SessionClientVersion = $i.SessionClientVersion
    $SessionStartTime = $i.SessionStartTime
    $DeliveryGroup = $i.DesktopGroupName
    
    write-host "Checking $VM"

    Write-Host "Checking $VM now" -ForegroundColor green
    write-host  "$AssetsLeft remaining to process.." -ForegroundColor cyan

    if (Test-Connection -ComputerName $VM -Count 1 -ErrorAction SilentlyContinue) {

        $Ping = "Online"
        
        write-host "$VM is online" -ForegroundColor Green
        
        $ID = Get-WinEvent -ComputerName $VM -FilterHashtable @{LogName='System' ; 'ID'='1058'} -ErrorAction SilentlyContinue | Select -last 1
        
        IF ($ID) {
            
            write-warning "Sys event 1058 found on $VM"
            
            $IDXML = [xml]$ID.ToXML()
            $DC = $IDXML.event.EventData.Data[6].'#text'

            $MSG = $ID  | Select -first 1 | Select-object -ExpandProperty Message
            $Time = $ID | Select -first 1 | Select-object -ExpandProperty TimeCreated
        }
        
         Else  {

            $MSG = "No event ID 1007 found"
            $Time = "N/A"
            $DC = "N/A"
        }       

    }
    
    Else {
        
        write-host "$VM is offline" -ForegroundColor yellow
        $Ping = "Offline"
        $ID = "N/A"
        $MSG = "N/A"
        $DC = "N/A"
        $Time = "N/A"

    }

    $OutArray += New-Object PSObject -property @{

    VM = $VM
    Ping = $Ping
    MSG = $MSG
    Time = $Time
    DC = $DC
    AssociatedUserFullNames = $AssociatedUserFullNames
    AssociatedUserNames = $AssociatedUserNames
    SessionClientName = $SessionClientName
    SessionClientVersion = $SessionClientVersion
    SessionStartTime = $SessionStartTime
    DeliveryGroup = $DeliveryGroup
    }

    $AssetsLeft --

} #ForEach asset

$OutArray | Select VM, Ping, Time, DC, MSG,  @{E={$_.AssociatedUserFullNames};Label='AssociatedUserFullName'}, @{E={$_.AssociatedUserNames};Label='AssociatedUserName'},`
SessionClientName,SessionClientVersion,SessionStartTime, DeliveryGroup | Export-Excel -Path "$ReportsPath\EventID-1058-$ReportTime.xlsx" -AutoFilter -Autosize -FreezeTopRow -BoldTopRow

$OutArray | Where {$_.MSG -like "The Processing*"} | Select VM, Ping, Time, DC, MSG, DeliveryGroup | ogv 

Buying a used car after 10 years out of the game

Intro

Here’s a non tech blog post

I love driving, maybe more than I realized when I first started test driving for a used car in May of this year. I’ve only owned 2 cars in my life, both Honda: A 1995 Civic Coupe, and a 2001 Civic EX, both were manual transmission. Prior to owning, I learned to drive manual transmission via my Toyota Corollas my parents had

My current interest in acquiring a used car comes from covid 19 boredom from last year. I live in Montreal, and from October 2020 to the spring of 2021, we had an embarrassing curfew of 9:30 pm. What better activity to just DRIVE AROUND, right?

As Montreal gets a lot of snow, an all-wheel drive car/cross-over seemed a good choice for the 5 months of the year we have hard packed snow. So, I started my search with Subaru Imprezas

However, as I’d not test driven a car since in 10 years, I decided to cluster my test drives, so, as much as possible, each time I went out for a test drive at a user / OEM dealer, I’d try out 2, 3, 4, 5 cars at a time

For my first 5 test drives, my friend Wayne – who already owns a beastly Mustang took me out – thanks man. For the other 15 test drives I did, I activated a car co-op membership (link)

To be honest, using this particular car co-op service served an excellent secondary purpose alongside the main service of delivering me from Montreal to the 3 major suburban areas where the dealers are located (Laval/West Island/South Shore). The secondary purpose , was the temp rental car serving as a “pace car”. 100% of the cars provided by communauto are hybrid/full electric, and some of these cars are excellent. They lack the luxury / power of 75% of the cars I’m driving, but in a few cases, I ended up liking my rental car MORE than the car I was going to drive. For instance, the Mercedes-Benz GLA 250! Lacked headroom, and was super boring to drive. The Toyota Corolla hybrid I used to get to the west island dealer actually had better torque in the lower gears, despite being almost 80 HP less than the GLA 250 on paper. WTF? Sorry, Merc fans, but I feel part of the appeal of this make is BRANDING.

Criteria

  1. Less than 200k KM
  2. $10k on the low, 20k on the high for price
  3. No rust on the body
  4. No weird body kits that will get destroyed in the winter
  5. No lowering
  6. No RWD cars that will be a chore to drive in the winter
  7. No prior accidents
  8. Non-CVT Automatic or manual only
  9. White/Black/Dark Blue/or in some cases, dark gray/silver for the exterior color
  10. Beige/Brown for the interior
  11. Back-up camera and Bluetooth are the only “deal breaker” accessory options I must have
  12. 38 inches or more of headroom. I have a peculiar driving style, where I prefer to sit VERY upright. I prefer this for safety reasons, as it gives me the best possible view for shoulder checks and front visibility. Because of this , the top of my hair brushes the cabin of the car when the head-room is less than inches. I’ve had to remove a few nice cars from the list due to cramped headroom: Merc-Benz GLA 250, Honda Civic 2014 Si

The below is the full list of test drives I did. Most were automatic / AWD. However, as time has gone on, I’m open to the idea of owning a FWD manual car again. If I could deal with winter driving in Winterpeg for most of my driving life, I can do it again here in Montreal. Where I was able to test drive 2,3,4,5 cars in a single day, I’ve shown in alternating colors of light / dark blue. The Infiniti EX35s were the only cars I wasn’t able to test drive alongside another car

  1. Kia forte 5 2017 – manual FWD: Mechanical issues, felt cheapt
  2. Subaru Impreza 2017 hatchback – Automatic AWD: Gutless and boring
  3. Audi A3 2013 hatchback – Automatic AWD: Fun, but no backup camera
  4. Infiniti ex 35 2010 black cross-over – Automatic AWD: Fun, but feels a bit old
  5. Infiniti ex 35 red 2013 cross-over– Automatic AWD: Fun, but feels a bit old
  6. Mercedes GLA 250 2017 cross-over – Automatic AWD: Head space issues, boring
  7. BMW X1 2017 cross-over – Automatic AWD: Fun, but over-priced
  8. Mazda 3 AWD 2019 sedan – Automatic AWD: Lots of fun
  9. Lexus IS 300 2017 – sedan – Automatic AWD: Awesome
  10. Acura ILX – sedan – FWD Automatic: Boring
  11. Infiniti Q50 2017 – Automatic AWD: Fun, but not as nice as Lexus IS 300
  12. Acura TLX 2017 – Automatic AWD: Fun, but only paddle shifters are lame
  13. Civic couple sport EX 2017 – FWD automatic: CVT, dang! Less fun than manual
  14. Volvo S60 T5 – i4 – Automatic AWD: Over-all, a top 4 choice
  15. Civic 2017 blue hatchback sport LX trim – manual FWD: Lots of un
  16. Subaru WRX -automatic AWD: CVT made this a boner killer
  17. Subaru WRX -manual AWD: Fun, but way over-priced
  18. Acura TLX 2015 – Automatic AWD: Down-shifting issues, no longer considering!
  19. Subaru Impreza – manual AWD: Gutless and weak @ 140 HP
  20. Honda civic SI 2017 – manual FWD: More fun than Civic 2017 sport as it’s got 204 HP
  21. Honda civic SI 2014 – manual FWD: Fun to drive, but removed from list due to only having 37.7 inches headroom
  22. Honda Civic sport 2018 – manual: AC unit wasn’t working in the unit I drove, which was brutal as it was 30 degrees Celsius outside at the time of my test drive
  23. Subaru WRX 2016 manual: Scheduled for Saturday Aug 14, 2021
  24. Toyota Corolla 2019 hatchback SE: It’s FWD, but omg, so much fun to drive with the upgraded 168 HP engine. I’m still keen AWD, but if one comes up for lease take-over or for a reasonable used price, i’ll consider it for sure
  25. 2020 Mazda 3 AWD Turbo hatchback: Amazing ride. Top of the pops for me now, however, I need to try out the same car with a 184 HP engine, the Boucherville, Quebec dealership I visited on Aug 19 only had the turbo to drive, which gives me a “feel” of the interior, and some of the handling features, but the 0-60 tests I did would certainly be different against the 184 HP engine common for this Make/model/trim/variant
  26. 2020 Mazda 3 AWD non-turbo 2.5 L hatchback: Sorry to say, it’s not even close to the turbo. As such, 2020 Mazda 3 2.5 L non-turbo HB’s are off my list. The price difference between the turbo/non-turbo 2019 > 2020 Turbos is about 10-12k, no thanks! 
  27. Infiniti G37X 2013: First time driving an older car in a while. yeah, it felt “old” for sure. I was keen, as it was a great price, around $10k cdn. However, the deal-breaker wasn’t the older infotainment system, it was head-room! I swear, my driving posture is not compatible with how a lot of cars are made! 

Here is a bullet list created from my test drive observations from May 2021 to now. If you’re in the market for a used car, perhaps you’ll find my notes useful

  • With an automatic transmission, having paddle shifters on the steering wheel or shift over-ride via the automatic gear shift is a must to keep things fun
  • Manual can be exciting work if HP is around 150-170, Civic Sport, however, it’s super boring for cars that are closer to 140 HP, Subaru Impreza
  • CVT based transmissions take the fun out of automatic
  • There’s not an easy way AWD during normal driving, I only had rainy conditions once, with the BWM X1
  • Close to 300 HP / 290 lbs torque is really exciting. The Lexus IS 300, Acura TLX, Infiniti Q50 all featured V6 engines close to this spec
  • Cabin head room is important, at 5’11, anything else than 37 inches was problematic for me
  • The Interior color and trim is more important than exterior, a black interior is really boring. Beige/brown is best
  • A backup camera is a must, I learned to reverse park / parallel park without one in the early 90’s, but backup cameras really do make for better parking
  • For whatever reason here in Quebec, there a lot of BMW X1/3 / 5 series that come in my KILLER exterior / interior color I covet: Dark blue outside/beige inside
  • A third-party cradle for smart phone is a must, ones that tactfully insert into the air vents are the best
  • USB is a must, GPS when driving is a KILLER on battery
  • The Toyota Corolla hybrid is an excellent pace car
  • Electric cars have a “pulse” like acceleration which is quite different. That being said, I can see the future, where a huge % of cars will be electric. I’m ok with it
  • Using consumer reports to review car reliability / recalls is a MUST. However, it’s not enough. For instance, the Acura TLX 2015 model I drove twice, has a serious issue with it’s double-clutch transmission, where it will randomly down shift or shift to neutral. This happened to me on the last test drive I did on a 2015 model, it’s not indicated on the related ConsumerReports.org entry for the Acura TLX 2015 onwards, but sure is here and here. I’ve removed the Acura TLX from my searches, if I want a “big dog” V6, it will be Lexus IS 350 or Volvo S60 T6
  • Any research you do for a potential used car needs to be offline (driving the damn thing), as well as reading owner feedback on car forums and reddit
  • Of the 20 test drives I did, I had to call ahead to book the time I would arrive. For 18/20 of those bookings, the dealer was NOT ready for me when I arrived: Car was sold, car was at another dealer, car wouldn’t start, car was dirty, front desk didn’t have my name. It was frustrating the first 5 times, then I made peace with it, as is the case in Quebec where customer service is generally terrible. As per the famous Maya Angelou quote, you sometimes need to accept the things you cannot change.
  • I drove 3 German flag ship cars: AUDI/BMW/Merc-Benz. For me at least, the higher cost / repair issues / maintenance cost aren’t worth it. I will focus my remaining search on Japanese cars
  • I didn’t drive a single American car. I could I guess, but I really just don’t feel like it. None came up recommended on review sites or recommendations from friends. If you know of one that fits nicely into the above, let me know!

Updates:

Aug 11, 2021 Shortlist
  1. Lexus IS 350 2014-2015
  2. Volvo S60 T5/T6 2015-2017:
  3. Honda Civic Si 10th gen 2017 onwards
  4. Mazda 3 AWD 2019-2020 – the 2.5 L 184 HP version
  5. Subaru WRX 2016 onwards – manual only with sport or sport tech

Aug 2, 2021:

During this week, I took a bunch of time to evaluate cheaper cars $10-14k. In each case, I had to remove them from further consideration, low HP / headroom issues

Aug 11, 2021

My search is now focused solely on cars that are $15-23k in price, and from the years 2014 onwards with absolute bare minimum headroom of 38 inches

Aug 12, 2021

Honda civic tenth gens have been removed my search! CR ratings are untypically low for a Honda Civic!

Air conditioner failures are common, and I noted this on my test drive yesterday of a 2017 Honda Civic sport. See #1 and #2

Civic FYM

Aug 19, 2021
 
I’ve removed the Lexus IS 350 2014 from the list. I do enjoy the exterior look and power of the V6 294 HP engine, but the interior feels a bit old against the other cars I’ve been driving. As such, here are the two models I’m now focusing my search on:
  • Mazda 3 AWD 2019 hatchback- the 2.5 L 184 HP version
  • Subaru WRX 2016 onwards – manual only with sport or sport tech

Aug 24, 2021

  1. Volvo S60 T5/T6 2015-2017:
    Removed Aug 24, 2021: These can be had for about $13-15k, but NONE feature backup cameras, and really, they look boring. For the same $$$ , I can get a nicer looking Infiniti Q50 / G37X with more HP/features, sorry Sweden, this car hunt is now ALL JAPANESE
  2. Mazda 3 AWD 2019-2020 – the 2.5 L 184 HP version
  3. Subaru WRX 2016 onwards – manual only with sport or sport tech
  4. Infiniti G37X 2011 – 2013:
    After some consideration, I should not be scared of buying an older Japanese luxury car. They can be had here in Quebec for about $10-12k. Feature 320+ HP engines with leather interior and backup camera + AWD! I enjoyed the Infiniti Q50 2018 I drove in July, so am now looking at it’s predecessor 

Aug 26, 2021

  • Infiniti G37X 2011 – 2013: Nope, too old and headroom issues!
  • As such, the last car standing is the mighty Subaru WRX 2016 onwards. I’m looking at used purchased around 20-22k CDN or ideally, a lease-take over via leasebusters.com or leasecosts.ca to get something a bit newer, and with warranty!

    Sustainable implementations

    I presented the following at at a MyCUGC event on May 18, 2021! Here’s a link to the recording

    I recently returned as a consultant for a client that I was working for in 2020, and it’s inspired this piece. The reason being, I realized I’d left some items in place that should have been removed and had confused the local staff.

    This is a common scenario; where you are the trusted expert for an implementation as an SME, but don’t fully hand off to other people on your team so they can continue to support your implementation when you’re gone (or on vacation).

    The BUS TEST

    Around 2014, I was working for a large financial company that was going full ITIL for their change management process. I was already familiar with change records, but the full ITIL compliance introduced a lot of extra steps that I found cumbersome. At one point, I got annoyed by having my change records rejected by a change managment person, so we decided to talk on the phone rather than bounce emails back and forth. He told me why my change records weren’t being approved: they didn’t pass the “BUS TEST.” He said: 

    “Owen, if you were hit by a BUS before the implementation start date, and were laid up in the hospital, would anyone else on your team be able to complete the change on your behalf?”  

    I had to answer “NO.” The instructions I included were written by me, for me, and were certainly not usable by anyone else. I went back, and re-wrote the instructions from an outsider’s perspective, and my change was approved. To this day, I follow the “BUS TEST” model for writing change MGMT records as well as documentation for clients / co-workers / blog posts. 

    With that MORBID intro out of the way, here are some common items of consideration for a successful & sustainable Citrix implementation:

    The dirty dozen (and then some)

    • Scheduled tasks that run under a non-service account: Don’t put in your OWN DOMAIN ID! I had to do this once, but I set a reminder for a future date to re-visit and set to a proper shared group service account.
    • Scheduled tasks that don’t contain any info on what they do via the description field, or link to external doc. When I used to work for a big financial institution years back, I’d host the accompanying docs on SharePoint, and link them in the description field.

    • PowerShell code: Other than the mighty Guy Leech, there are few of us who are formally trained in programming, but, we are writing code whenever we open Powershell ISE or MS Visual Studio Code to create PowerShell code.

      I’m NOT an expert on PowerShell code, I’d consider myself “intermediate.” One thing I’m good at, is documenting! Document your code inside the script, or in greater detail via the README on github. Don’t assume everyone understands your code. People will run into an error, not report it (especially if they don’t know you) and never try to use it again. This happened to me with a health check script I wrote last year. V1 required more exit statements at the start for restricted environments, and I only caught it when someone working for the same company as me reported it. The Internet is like space: no one can hear you scream. People won’t report issues and will move on to the next thing they find by search engine search. Generally speaking, you should assume that any code you write for a client, is code that you “own” and ultimately have some level of responsibility should it stop working as expected for the clients who continue to use it.
    • Related to the above point ; scripts that don’t include a synopsis / header to explain their function, or, for more complex scripts, no accompanying documentation in wiki/SharePoint/OneNote/etc. Create a README on your git hub or at least include what the script does in contents of the script
    • If you’ve got the time, it’s a good idea to leave the client with a basic means to health check the Citrix / hosting resources you setup for them: Shameless plug, I’ve got two scripts that work for this exact purpose: One is fully documented on my personal blog here, and another for health checking vCenter ESXi environments is on my git hub here . Both of these scripts require very little config to run, and give you a quick overview of your environment that you can once, or as a scheduled task 
    • GPO’s and OU’s that were used for testing called “TEST,” “TST” that don’t contain any active items. If they ain’t used, delete ’em!
    • A GPO for each setting/settings group: This comes down to preference, I suppose, but for me, I like to have large sets of common EUC-related settings in just two GPOs: One for computer, one for user-side settings. Having lots of GPOs can be confusing if they aren’t labeled, and can adversely impact logon times as each GPO must be processed at user logon / computer start-up
    • Not commenting your GPOs/GPP entries: With Citrix, there are obscure registry keys that affect user experience that aren’t well documented, CTX hooks for instance! Always use the comments field in your GPO/GPP settings to link to the blog/KB where you got it, and explain why you are using it. If it’s only supposed to be in place temporarily , make a note that it’s to be removed at date “MM/DD/YYYY”
    • Use of un-supported settings/registry keys/etc. that were in place to resolve an issue, that were later resolved by a patch from the vendor. I’ll admit, this one isn’t easy to track. I’m open to ideas on how you’ve re-visited old registry keys/GPO settings for clients you’ve set up.
    • For work-arounds you provide to fix symptomatic issues, keep in mind, “quick and dirty” can get dirtier as time goes on. Ever forgotten a banana peel behind your trash can? I had a job once years ago, where PRPs (problem resolution procedures) that had beginning / end date! When the end date came up, you had to evaluate if the work-around was still valid, and justify renewal of the PRP. This was a great process, as you really shouldn’t be using a work-around for more than a few months, right!?
    • Citrix products that introduce extra infra components / agents: WEM is a popular component that provides an alternative to the normal method of applying user settings to VDA sessions: GPO/GPPs. However, if you aren’t using the performance optimization features of the WEM agent, the use of WEM to control your user experience such as resetting desktop icons/mapped drives/printers introduces two caveats: 

      1: On-site staff who are already familiar with GPOs/GPPs will need to learn the new WEM console mgmt interface.

      2: The WEM agent is another binary to update in your golden image / platform layer. There is no LTSR for WEM, and each release introduces a risk of a new “bug” that can be user experience impacting.

      3: If you’re using WEM on-prem, you’ll have to maintain an SQL instance as well as the windows server instance where your WEM instance is installed. If/when either of these components suffers an outage, the WEM agents will go into “cache” mode, and you won’t be able to apply new settings to your WEM agents.

      For my clients that have engaged me in 2020/2021, and already have GPOs in place to control the base required user experience, I’m recommending they only use WEM via Citrix Cloud for it’s “killer feature,” which is its CPU/memory management.
    • App-layering: The inspiration for this presentation came from a project I was leading in 2019 to migrate a client on a Win 7 VDI with PvD to Win 10. The original plan was to deploy app-layering + user layers as “like for like” replacement for the depreciated PvD feature. The POC using Citrix App layering was successful, but process docs we had written for the client were so complex / long, I knew the client would be challenged to continue to support the implementation once I was gone. Instead, we went with a solution that had ALL the key apps installed on one golden image, and used Microsoft FSLogix app masking to selectively hide/un-hide apps based on AD group membership. The project was a complete success. We were able to complete on-time, despite staff shortages due to covid budget restraints. 
       
    • PVS vs MCS: I started using PVS in 2016, and MCS in 2019. I’m familiar enough with both products, and am happy to work with either, but the administrative overhead of PVS is significantly higher with PVS. If your client’s local staff aren’t dedicated to Citrix, which is often the case, PVS is not going to be the right choice for them. As well, MCS provides on-prem clients a path to Citrix Cloud, PVS as of the time of this writing does not

      I’ve created two scenarios, one where you have lots of time towards the end of a project for documentation, the other where you’ve got very little time

      Short amount of time

      -AD Group > Delivery group mapping
      -Golden image virtual machine names > AD machine account mapping
      -Inventory of on-prem Citrix components and service account names
      -Storefront internal / external URLs

      Greater amounts of time
      -All of the above items
      -MCS / PVS / App Layering image update process
      -Full documentation on scheduled tasks
      -Full documentation on user migration scripts
      -Architecture diagrams / connection flows
      -Problem resolution procedures for the helpdesk 

    The above list is culled from my work as a Citrix consultant here in Montreal, Quebec, Canada and from my past experience working for large financial companies. Your own experience will vary.

    For me, the key to a successful implementation isn’t just about checking off the all the project requirements, but ensuring that the platform you deliver will continue to work a day after you’re gone, and a year later!

    Here are some good litmus tests you can use to gauge staff acceptance during the course of the project implementation:

    • If you’ve already written process docs, have the local staff read/reviewed/understood what’s written? Are they capable of completing basic changes when you’re not gone? Sick days, holidays, etc.
    • How many emails do you get asking questions you had previously answered?  
    • Have you ever been called on a day off for assistance on your implementation? 

    If any of the above come up with any regularity, you might have created a monster that only YOU can tame. The client will call you back, your sales folks might be happy as you’ve increased billable hours, but your professional rep (and your company brand) may take a hit



    Citrix CTA for 2021!

    Towards the beginning of the pandemic, I had read some interesting discussions about “staying productive” during the pandemic. At the time, the discussion I was following was on twitter. As you might be aware, no discussion is ever FINAL on twitter. Some folks believed 2020 was a good time to learn a new language, learn to do your own oil filter changes, learn to code, cut your own hair, etc, etc. Others lamented those who could not stay at home perpetually. As I often am with most divisive topics, I was split. I’ve been using Microsoft OneNote to keep my life in order for over ten years, could I check off a few more boxes in 2020? The answer was YES

    For 2020, I managed to contribute some good longer blog posts, and was WAY more active on Slack / twitter than in years gone by. I also did a presentation for MyCUGC on the use of Packer to automate windows golden image creation, which was a lot of fun. However, I also had times where I wasn’t as motivated, but I kept my head up, dealt with the gray sky skies by increasing my vitamin D intake and drank a lot of coffee, maybe a bit too much :p

    After some consideration, I applied for the Citrix CTA program in October of 2020.

    I was inspired to apply for the program, through my friendship with Jonathan Pitre, Canada’s only other CTA . “Why do we only have one, CTA in Canada? ” I used to ask him. No more! Jon is truly an inspiration when it comes to end-user computing / virtualization , if you don’t already follow him on twitter bird, do it now! Even better, favorite his GIT hub, he’s got some incredibly useful PowerShell scripts for automating app installs on your EUC golden images on his GitHUB.

    Shameless, plug, Jon/I both work for the same Quebec-based company: https://iti.ca

    This January, I was admitted into the CTA program! I’m super happy to be involved, and am very much looking forward to contributing more to the community as time goes on

    Announcing the new 2021 Citrix Technology Advocate (CTA) awardees! | Citrix Blogs

    Passed VMware VCP DCV 2021!

    On Thursday March 4, 2021, I sat/passed the VMware VCP DCV 2021

    My path to this certification was a bit LOOOONG, like this here cat

    My path to the VMware VCP DCV cert / status started all the way back in 2019. I didn’t actually start studying in 2019, but I was given this pin from my former co-worker Pascal Proulx after he attended the VMWorld 2019 event. 

    In 2020, after passing the vSphere 6.7 foundations exam, I placed the below pin on top of my HP mini computer as a reminder to write / pass the VCP DCV. I didn’t wear the pin on my clothes outside, that would be faking! It remained at home next to my stuffed animals and other desk trinkets I’ve been using to keep me SANE during the various rounds of covid lockdown here in Montreal, Quebec.

    At the time of this blog post, the official requirements from VMware stipulate you must sit in a VMware academy accredited institution . Which I did twice actually, once on the VMware VCP 6.5 track, then again in 2020 on the VMware VCP 6.7 track via an excellent school in North Carolina called Stanly College .

    • But here’s the thing, VMware recently announced EOL for the VCP DCV on the 6.7 track, you can STILL book it via Pearson-VUE, and they don’t actually warn you that the cert will EOL by July of 2021. 2 weeks back, I was ready to sit the exam, and was on the final page of the related Pearson VUE exam page portal, just before I entered in my credit card details , I decided to check the EOL dates for the 6.7 exam track (2V0-21.19) and found the following blog post. I’ve been using vSphere 7 (vCenter / ESXi) for almost a year now, so after reviewing some additional study materials and practice exams, I was comfortable enough to sit the updated exam based on vSphere 7 2V0-21.20. I took some time to enable/learn some of the exam topics on my home lab:

    • vCenter HA: I kept this running for a week, but found it over-kill for my home lab which at the time was 4 ESXi nodes, now 3. For now, if I want to ensure my vCenter instance continues to run while I do maintenance on an ESXi host, I just vMotioned it over my 10 GBe network to another online ESXi host

    • vSphere fault tolerance: This feature is straight up MAGIC. Do they hand out tech innovation / invention awards each year? I stopped watching the Academy awards more than ten years ago, I think we should have an Academy awards for tech, and a retroactive award for this feature. 

    • 2 and 3 node vSan setups: In 2020, I was running a 3 node vSan cluster , in 2021, I switched to a 2 node closer with an external vSan witness appliance. Simple/ez/fast!

    To close ; I had read online a few times that the VCP DCV exam has had 50% failure rate for a few years, I gotta say, it’s true, it’s a challenging but worthwhile exam to write. I learned a TON and am much more confident to handle service and project requests in the vSphere realm

    And more good news I just found today, no more time-bombs on folks that write the exams! 

    Have a good weekend, all!

    ID windows evaluation licenses time bombs!

    I’ve been using a PowerShell script to check my home network/lab assets for many years, as time has gone on, and I’ve added and removed functions from it. Additional details are available on this post from last year:

    Powershell health check script – GetVpro (wordpress.com)

    Sometime in the past few years, I removed a function I had to check for remaining evaluation license days on windows server / desktop editions. I had removed it, as like a lot of lab setups, my assets are in a state of flux. I’ll stand-up a PVS , file server or other windows asset for learning, or to help trial something for a client, then I’ll delete the VM(s) 

    As time has gone on, I’ve been the benefit of keeping some assets up FOREVER, such as my domain controllers. For these, I’ve acquired legit keys from past employers or via the buy/sell market. As of last week, I had thought that all my key infra windows servers were licensed, but noted my second domain controller was offline. As you may be aware, when you hit the end of a windows evaluation license , the asset will boot, remain up, then gracefully shutdown after a period of time.

    Monday morning, my daily 10:30 AM Get-AssetHealth report showed me my second DC was offline, I powered it on, made some coffee, came back later on, and noted she was offline again! Logging on confirmed the issue, 0 days left on the eval , oops!

    I went back to my list of spare keys, and quickly converted the trial to a legit install via the following one-liner:

    DISM /online /Set-Edition:ServerStandard /ProductKey:YOUR-LEGIT-KEY-12345 /AcceptEula
    

    Crisis averted, but what other treasures await on my other 20+ VMs at home? Time to add back the Get-LicensedDays remaining function to my Get-AssetHealth script! Here it is:

    Function Get-LicDaysRemain {
        Param ($Asset)
        
        $LicDaysRem = Get-CimInstance SoftwareLicensingProduct -ComputerName $Asset -Filter "ApplicationID = '55c92734-d682-4d71-983e-d6ec3f16059f'" | Where-Object -FilterScript {$_.LicenseFamily -like "*eval*"} `
        | Select-object -expand GracePeriodRemaining
    
        IF ($LicDaysRem -ne $Null) {  
        
            $LicDaysRem = new-timespan -minutes $LicDaysRem | Select-object -ExpandProperty Days    
            Write-warning "Remaining licensing days left = $LicDaysRem"
    
        }
    
        Else {
    
            $LicDaysRem = "Valid license installed"
            
    
        }
        
        Return $LicDaysRem
    }
    

    Running the update Get-AssetHealth script, I found another 5 assets that are set to time-out their windows server / desktop evaluations in the next month.

    However, they aren’t critical to home lab infra, so I’ll probably just re-build them using a golden image, or packer which I learned last year

    Owen

    Random fix: Resolving issues with Chrome pinned icons associated to 32-bit chrome where 64-bit is now installed

    Season’s greetings!

    Here’s an interesting fix I found to an odd issue from last week at a client

    It should be noted this issue could probably be fixed with Avanti, or WEM, but that wasn’t an option for my client, so I just used PowerShell instead

    For this client, we had been pushing a managed start menu that included pinned icons in the TILE area as well as a set of pinned task bar area icons. One of those pinned icons was Chrome

    As many of you are aware, Chrome has been 64-bit for years, but in their wizDOOM, the Chrome dev folks install to c:\Program Files (x86). This was reported as a bug to Google six years back, they finally “fixed” it this year (2020). Damn, son

    Google Chrome is soon going to be installed in a different directory on Windows – gHacks Tech News

    So! Versions from June 2020 onwards will install to c:\Program Files , instead of c:\Program Files (x86)

    In my testing on a Win 10 VDI golden image, I immediately noted the pinned icon for Chrome had turned into a white icon, indicating the path was broken. Now, the tricky thing, these pinned icons are PER-USER, and located here:

    %AppData%\Roamin\Microsoft\Internet Explorer\Quick Launch\User Pinned\Taskbar

    The Win 10 Citrix VDI deployment I’m working on started late last year, so we’ve got 1000’s of users with potentially bad cached pinned start menu icons affected as soon as the golden image is updated, and the Chrome install path flips from c:\Program Files (x86) to c:\Program Files

    The fix was to use the following Powershell code to check if the user was logging into Win 10 Citrix VDI that had been updated with Chrome installed to c:\Program Files, and check against their pinned shortcuts , and take action as required to reset their pinned icons from local system defaults

     

    $WScript = New-Object -ComObject WScript.Shell
    
    if (test-path "C:\Program Files\Google\Chrome\Application\chrome.exe") {
    
        $PinnedLNK = Get-Childitem "$Env:Appdata\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar" | Where {$_.Name -like "*Chrome*"}
                
        ForEach ($i in $PinnedLNK) {    
    
            $Fullname = $i.FullName
            $Shortname = $i.Name
            $LNK = $WScript.CreateShortcut($i.FullName)
    
            write-host "Checking shortcut paths for 32-bit Chrome" -ForegroundColor Cyan
        
                If (($LNK.WorkingDirectory -eq "C:\Program Files (x86)\Google\Chrome\Application")) {        
                        
                    Write-Warning "32-bit chrome pinned LNK detected where 64-bit Chrome is installed, starting pinned icon reset process"
    
                    remove-item $Fullname
    
                    copy-item -Path "C:\Users\Default\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\Taskbar\Google Chrome.lnk" -Destination "$Env:AppData\Microsoft\Internet Explorer\Quick Launch\User Pinned\Taskbar\" -Force
    
                    Remove-Item 'HKCU:\Software\Microsoft\Windows\CurrentVersion\CloudStore\Store\Cache\DefaultAccount\*$start.tilegrid$windows.data.curatedtilecollection.tilecollection' -force -recurse -ErrorAction SilentlyContinue
                
                    get-process shellexperiencehost -ErrorAction SilentlyContinue | stop-process -force -ErrorAction SilentlyContinue
        
                    write-host "Pausing for for 3 seconds..."
                    start-sleep -s 3
        
                    remove-item -Path "$env:LOCALAPPDATA\Packages\Microsoft.Windows.ShellExperienceHost_cw5n1h2txyewy\TempState\StartUnifiedTileModelCache.dat" -Force -ErrorAction SilentlyContinue
                    remove-item -Path "$env:LOCALAPPDATA\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\TempState\StartUnifiedTileModelCache.dat" -Force -ErrorAction SilentlyContinue
                    remove-item -Path "HKCU:\\Software\Microsoft\Windows\CurrentVersion\Explorer\Taskband" -Force -Recurse -ErrorAction SilentlyContinue
    
                    Get-Process Explorer | Stop-Process -force -ErrorAction SilentlyContinue
            
                }
    
                Else {
    
                    Write-host "No action required" -ForegroundColor Cyan
    
                }    
         } #ForEach LNK
    
    } # If test-path on "C:\Program Files\Google\Chrome\Application\chrome.exe"
    
    Else {
    
        write-host "Only 32-bit chrome is installed on this system" -ForegroundColor Cyan
    
    } 
    

    10 min Windows 10 / Server 2019 build automation via OSDBuilder, autounattend.xml and Packer.IO

    Intro

    As IT pros, we’ve got no shortage of new / interesting and useful tools available to us. With the adoption of open-source software into consumer and enterprise environments, a lot of the tools are free! As long as you don’t need serious 24/7 support that is

    How do you know which tools to use? COMMUNITY ! For me, that introduction came in 2017, I met Jonathan Pitre (twitter) while working in a previous job. Jon is Canada’s only CTA and has become a good friend since I met him in 2017. He opened my eyes to the “power of the community” that exists for EUC. Previously, I was aware of the EUC/CTA/CTP community by way of those awesome posts from the two Carl’s, but I wasn’t in the habit of regularly digesting the related blogs or going to events like Synergy or the like. Jon changed that REAL QUICK by letting me import his RSS blog list and also listed off a bunch of interesting tools he had used or was made aware of via the EUC community.

    So, THANK YOU JON !!

    This week, I checked another box off from Jon’s original list a few years back, Packer.io, and I’m glad I did. It’s amazing software. Also this week, I took https://osdbuilder.osdeploy.com for a spin, which provides a way to replace legacy dism commands and MDT

    The challenges we face

    Building windows 10 / server 201x images for cloning has the following challenges (at least as far as I’ve encountered)

    1. Human-error by hand cranking settings @ the virtual shell level
    2. Human-error by hand cranking settings post windows-build
    3. The amount of time it takes to apply windows updates once the base build is done
    4. Time that’s lost while waiting for steps 1/2/3 to complete

    Each of the above challenges can be solved by combining a few simple tools that I will describe below

    The solution

    The solution I’ve found that worked for me is based on the open-source tool Packer

    There’s a lot of marchitecture/jargon/nerd speak out there at the moment around dev-ops, CI/CD, “infrastructure as code. I’ve read plenty of posts online where the conversations contained so many acronyms, it made me question my command of the English language. It’s about time-savings and ensuring build consistency

    The below steps are based on using Packer with vSphere, as it’s still the most common hypervisor for deploying Citrix workloads, and it happens to be what I use @ home on my VMUG licensed 3 node vSan cluster (<>shout out</>) There are official and community templates for Nutanix as well as all the major cloud platforms, but I’m only going to speak to what I know for this blog post, Packer + vSphere

    When doing the work this week, I found that some of the reference blog pages and associated config files (.JSON/XML) for use with Packer had not been updated in a few years, so I definitely lost some time while troubleshooting, but, that’s how you learn! I can’t blame the authors of these blog posts ; how often do you go back and update your own blog posts? I’m guilty of this as well. If you’re reading this in the FUTURE and find that anything I’ve written below doesn’t work for you, please let me know in the comments or message me on twitter bird or email, I’ll try and help ya out! That’s community , baby !

    Let’s get started with the proposed tools and proposed workflow at each stage to solve each of the 4 challenges I’ve listed above

    Challenge 1: Standardizing VM shell settings
    Here is our first case for use of the Packer tool. Packer uses .JSON based config files. These can be used to select the common shell values you select when manually creating a shell via the vSphere HTML5 client, or PowerCLI

    Challenge 2: Standardizing windows install settings
    Here, we will be using Microsoft autounattend.XML templates to pre-fill all the important settings that we would normally have to click / mouse through in a standard windows install.

    Use of autounattend.XML files isn’t something new, but there are integration pieces that are possible by way of Packer integration which makes the autounttend.xml a lot more useful. You can use the Microsoft SDK and the Windows System image manager, but I’ve also got templates for Win 10 / Win 2019 server ready to use on my GIT hub HERE to save you having to create your own

    Challenge 3: Windows updates last longer than it takes to make a drink a cup of coffee
    OSDBuilder will be used to slipsteam in windows updates to our base ISO. As we are slipstreaming an offline image, we no longer need to reboot the live un-patched / out-of-date windows OS a bunch of times while we wait for each update to apply. This is a huge time-saver once combined with packer to attach the updated ISO

    Challenge 4 – the time lost by waiting for each of the above 3 processes to complete
    Again, Packer is here to help us out as the GLUE that ties it together! Packer is used to start step 1 and step 2, and thus fixes the issue with time lost in-between these steps

    Detailed steps

    Packer install/config

    • Download Windows 10
      / Server 2019 trials from Microsoft
    • Download a recent copy of VMwareools.iso for windows
    • Extract the VMware tools .zip , and rename windows.iso > vmwaretools.iso and upload to your preferred vSphere datastore folder
    • Refer to the following YouTube video on how to create a slipstreamed ISO for Win 10 or Server 2019 using OSDBuilder
    • Upload the slipstreamed / updated windows ISO(s) you created in step 2 for either Win 10 or Server 2019 to the same vSphere datastore as used in step 3The paths for these ISOs will be entered in your packer .JSON later
    • Download the packer.exe from packer.IO HEREI found blog posts from 2017/2018 that reference the requirement to download 3rd party plug-ins for the vpshere APO. These are no longer required! The dev folks from the packer team integrated support for the vpshere API into the main packer.exe in Feb of 2020 (see GitHub post here)
    • Extract it to c:\Program Files\Packer on your preferred build machine. I’m re-using an existing “build” machine that I use where I’ve got the Windows deployment toolkit / PDQ deploy installed
    • Start > Run > Sysdm.cpl > Advanced > Environment variables > SYSTEM VARIABLES > PATH > EDIT

    • Add the path to c:\Program Files\Packer
    • Open my Build-Packer git hub repo HERE, download everything as a ZIP to your desktop
    • Extract the contents with your favorite zip manager, and open the folder Build-Packer-master
    • Move or copy the config/scripts folders to c:\Program Files\Packer

    Windows Autounattnend.xml amendments

    • Install / open Microsoft Visual Studio code
    • For Server 2019 open:
      c:\Program Files\Packer\Config\Autounattend\Server_2019\autounattend.xml
    • For Win 10 open:
      c:\Program Files\Packer\Config\Autounattend\Win10\autounattend.xml
    • For the below edits, I’ve not included the line number, as it might change as updates to the related GIT hub repo are made. In all cases, use CTRL-F to search for the blow of text highlighted in yellow to find the entry
    • First, you will need to edit the entries for your preferred language
    • Amend the below line if you want a larger C drive size, just remember to amend within the associated packer .JSON file as well, covered in the later steps
    • NOTE: From experience, and especially true with golden images for later use with RDS/Citrix worker roles, it’s best to do your base build in English (en-US) and add your local language after the fact via a language pack
      <SetupUILanguage>
      <UILanguage>en-US</UILanguage>
      </SetupUILanguage>
      <InputLocale>en-US</InputLocale>
      <SystemLocale>en-US</SystemLocale>
      <UILanguage>en-US</UILanguage>
      <UserLocale>en-US</UserLocale
    • Amend the below line if you want a larger C drive size, just remember to amend within the associated packer .JSON file as well, covered in the later steps
      <CreatePartition wcm:action=”add”>
      <Order>1</Order>
      <Size>50</Size>
      <Type>Primary</Type>
      </CreatePartition>
    • I’m using trial versions of Win 10 / Win 2019 in my lab, if you want to use licensed versions and plan on using KMS (Microsoft reference site) on your network to activate them, uncomment the below line. If you input a KMS key that doesn’t match your initial media, your build will fail. Yes, I made this mistake on my first day of building<ProductKey>
      <!– <Key>6XBNX-4JQGW-QX6QG-74P76-72V67</Key> –>
      <WillShowUI>OnError</WillShowUI>
      </ProductKey>
    • Amend the below lines to include you preferred account name associated to the build, and org

      <FullName>Win 2019 Packer Build</FullName>
      <Organization>Your Org Name</Organization>

    • Amend your time zone for wherever you are in the world

      <TimeZone>Eastern Standard Time</TimeZone>

    • Amend the computer name as you see fit

      <ComputerName>w2k19-packer</ComputerName>

    • Change the below value as you see fit, else, you’ll be left with the default password, which is insecure by virtue of being shared on GITHUB
      <AutoLogon>
      <Password>
      <Value>20SuperSecure3d20</Value>
      <PlainText>true</PlainText>
      </Password>
      <LogonCount>2</LogonCount>
      <Username>Administrator</Username>
      <Enabled>true</Enabled>
      </AutoLogon>
    • Match the password set in step 23 in the below section
      <AdministratorPassword>
      <Value>20SuperSecure3d20</Value>
      <PlainText>true</PlainText>
      </AdministratorPassword>
    • Note: Record the password used here, as it will be re-used in the related PACKER JSON file which we will edit when done with the autounattend.xml file
    • Save the file. As well, I would suggest backing it up to a network location for safe keeping, the XML/JSON configs are the most important files in this process, if you corrupted or lost either at any point, it’s probably the most time consuming part of this entire process, and it’s no fun to have to re-create them. I’m saying this, as it happened to me I was writing this blog post

    Packer JSON config amendments

    • Using Notepad ++ or MS Visual Studio code, open the PACKER.JSON for Win 10 / Server 2019
    • If you want to use a different structure for your source scripts/config files, you will need to amend the following section under floppy_files
      Otherwise, I would suggest leaving it. My own JSON template was culled from other templates for packer vpshere I found online, they all followed a similar formatting

      “floppy_files”: [

      “config/Autounattend/Win10/autounattend.xml”,
      “scripts/Start-FirstSteps.ps1”,
      “scripts/Install-VMTools.ps1”,
      “scripts/Start-DomainJoin.ps1”,
      “scripts/Enable-WinRM.ps1”
      ]

    • If you want to convert your completed VM/windows install to a template at the end of the build, you can set the following value to “true”

      “convert_to_template”: “false“,

    • Amend the path to vSphere datastore where you uploaded the vmwaretools.iso to in a previous step

      “iso_paths”: [
      “{{user `os_iso_path`}}”,
      “CHANGE ME] CHANGE ME/vmwaretools.iso”
      ]

    • Edit variables section at the end of the file to match your environment
      TIP: Having the “os_iso_path” on the same datastore where you VM will be created will speed up the process, same goes for the vmwaretools.iso

      “variables”: {
      “os_iso_path”: “[CHANGE ME] CHANGE ME/WIN10_2004.ISO”,
      “vm-cpu-num”: “2”,
      “vm-disk-size”: “40960”,
      “vm-mem-size”: “4096”,
      “vm-name”: “WIN10-Packer”,
      “vsphere-cluster”: “CHANGE ME”,
      “vsphere-datacenter”: “CHANGE ME,
      “vsphere-datastore”: “CHANGE ME”,
      “vsphere-folder”: “CHANGE ME”,
      “vsphere-network”: “VM Network”,
      “vsphere-password”: “CHANGE ME”,
      “vsphere-server”: “CHANGE ME”,
      “vsphere-user”: “administrator@CHANGE ME”,
      “winadmin-password”: “CHANGE ME”
      }

    NOTE: All guest OS types are listed HERE

    Start Packer Build

    • With the above XML/JSON files saved, it’s time to start the build
    • On your build machine, open an admin PowerShell prompt and CD over the directory you installed packer and your config/script files to CD C:\Program Files\Packer
    • Enable logging by the following two commands:
      $env:PACKER_LOG=1
      $env:PACKER_LOG_PATH=”packerlog.txt”
    • Start the build by running the following for your chosen OS
      Server:

      Packer.exe build config/json/server_2019.json
      Win10:
      Packer.exe build config/json/win_10.json
      TIP: I found it useful to watch the process on two screens. Left screen where you’ve got your packer process running, right screen where you’re logged into vSphere

    • The Packer.exe process should connect to your vSphere instance via an API call via the credentials you have in the JSON file you edited in the previous section
    • Within vSphere you should see the new VM shell should be created within just a few seconds based on the settings you chose for vCPU, RAM, HDD size on datastore you choose, and the network you chose
    • The new shell will have the VMware tools ISO attached, as well as either the win_10 / server_2019.ISO you uploaded in step 1 and defined in the .JSON config file
    • A virtual floppy drive will be created on the remote shell, and all those scripts listed in the [floppy_files] section of the .JSON config that correspond to .ps1 scripts in “scripts” sub-folder on the machine where you’re running the packer.exe will be copied over to the remote shell
    • The VM will boot up and be start the windows install via the ISO you specified
    • Here is what you should see on the machine running where you started the packer.exe process, Here is what you should see while you wait for windows to install

      vsphere-iso: Creating VM…
      vsphere-iso: Customizing hardware…
      vsphere-iso: Mounting ISO images…
      vsphere-iso: Creating floppy disk…
      vsphere-iso: Copying files flatly from floppy_files
      vsphere-iso: Copying file: autounattend.xml
      vsphere-iso: Copying file: scripts/Install-VMTools.ps1
      vsphere-iso: Copying file: scripts/Start-FirstSteps.ps1
      vsphere-iso: Done copying files from floppy_files
      vsphere-iso: Collecting paths from floppy_dirs
      vsphere-iso: Resulting paths from floppy_dirs : []
      vsphere-iso: Done copying paths from floppy_dirs
      vsphere-iso: Uploading created floppy image
      vsphere-iso: Adding generated Floppy…
      vsphere-iso: Set boot order temporary…
      vsphere-iso: Power on VM…
      vsphere-iso: Waiting for IP…

    • The Windows setup is hard-coded to scan all removable drive sources for an autounattend.XML. In this case, we’ve got the floppy drive mounted, so A:\autounattend.xml will be located and parsed
    • As long as there aren’t any type-o’s or formatting errors from editing the autounattend.xml for your environment, the windows install should be completely automated all the way to the desktop logon. In my own environment, the build process takes about 9 minutes, which I’m quite happy with. It’s faster than it takes me to take my dog outside, which is a good test
    • Based on the <autologon> that was set within the autounattend.xml file template I provided on git hub, the newly created windows VM will logon and start processing the various <SynchronousCommand> entries listed.
    • Of these commands, there are 4 entries that are hard-coded to run the .ps1 scripts from the virtual A:\ drive that was created earlier in the process. They are as follows:
    • Start-FirstSteps.ps1 will run and go through some basic packer > windows integration steps that I consolidated from the Guillermo Musumeci’s GIT REPO
    • The second script Install-VMTools.ps1 is critical, and I spent some time this week to refine the process by which VMware Tools is installed on a windows machine. It’s been a long-standing issue where the windows VMware tools package will install the drivers correctly when called using:
      Start-Process “setup64.exe” -ArgumentList ‘/s /v “/qb REBOOT=R”‘ -Wait
    • However!!!! the actual windows service called “VMWARE tools” which packer requires to finish the build will sometimes fail to install on the first attempt. To get around this, I found a great script which has logic to detect for this scenario and re-install VMware tools as required. Thanks Tim ! My script has a few edits which I made for use with my own packer builds
    • The third script is Start-DomainJoin.ps1. This script doesn’t actually start the AD domain join process, rather, it copies the script to a new directory called c:\scripts on the newly created windows install, and creates on a shortcut on the desktop called “Join Active Directory”

      This can be called later if you choose to actually join your AD environment, you would just need to enter in valid AD domain admin creds and a domain name, once it’s done, it will delete the shortcut on the admin desktop

    • The final script processed by our autounattend.xml is Enable-WinRM.ps1. Like the Install-VMtools.ps1 script, this one is critical for packer integration. WinRM is used from the remote packer instance to the newly created shell / windows install to finalize the build
    • Once the Enable-WinRM.ps1 script has completed, your running packer.exe process should recognize the remote IP of the newly created VMware shell, and initiate the final steps, which is to disconnect the floppy/CDROM drives and power off the VM
    • Here is what you should see towards the end of the process for a successful build

      vSphere-iso: IP address: 192.168.1.172
      vSphere-iso: Using winrm communicator to connect: 192.168.1.172
      vSphere-iso: Waiting for WinRM to become available…
      vSphere-iso: WinRM connected
      vsphere-iso: Connected to WinRM!
      Vsphere-iso: Provisioning with windows-shell…
      vsphere-iso: Provisioning with shell script: C:\Users\admin\AppData\Local\Temp\windows-shell-provisioner225687659
      vsphere-iso: C:\Users\administrator\ipconfig
      vsphere-iso: Windows IP Configuration
      vsphere-iso: Ethernet adapter Ethernet:
      vsphere-iso: Connection-specific DNS Suffix . : yourdomain.LOC
      vsphere-iso: Link-local Ipv6 Address . . . . . : fe80::18e2:5f3a:617e:8573%6
      vsphere-iso: Ipv4 Address. . . . . . . . . . . : 192.168.1.172
      vsphere-iso: Subnet Mask . . . . . . . . . . . : 255.255.255.0
      vsphere-iso: Default Gateway . . . . . . . . . : 192.168.1.10
      vsphere-iso: Shutting down VM…
      vsphere-iso: Deleting Floppy drives…
      vsphere-iso: Deleting Floppy image…
      vsphere-iso: Eject CD-ROM drives…
      vsphere-iso: Clear boot order…
      Build ‘vsphere-iso’ finished.

    • That’s it! You can boot the VM up again for the next stage of whatever build steps you want to do. Join to the domain, run windows updates, install language packs or convert it to a template for re-use creating more clones, but you’re basic build should now be completed

    Troubleshooting

    Failures that occur with the PACKER JSON FILE

    • Ensure that you’ve got your floppy_files section formatted correctly based on the folder structureDon’t forget any “,” on your lines! The packer.exe will tell you, and give the line reference, ensure you’re doing your edits in a proper editor, Visual studio code or Notepad ++
    • Ensure you’ve set ALL of the below correctly
    • The below 5 files should be saved to like-named folders where you packer.exe is extracted to. If they are missing, packer will bail out

      “floppy_files”: [
      “config/Autounattend/Win10/autounattend.xml”,
      “scripts/Start-FirstSteps.ps1”,
      “scripts/Install-VMTools.ps1”,
      “scripts/Start-DomainJoin.ps1”,
      “scripts/Enable-WinRM.ps1”
      ]

    • The below values are case-sensitive , ensure you match them exactly as they are set within vpshere. I would suggest using TWO screens for this part

      “variables”: {
      “os_iso_path”: “[CHANGE ME] CHANGE ME/WIN10_2004.ISO”,
      “vm-cpu-num”: “2”,
      “vm-disk-size”: “40960”,
      “vm-mem-size”: “4096”,
      “vm-name”: “WIN10-Packer”,
      “vsphere-cluster”: “CHANGE ME”,
      “vsphere-datacenter”: “CHANGE ME,
      “vsphere-datastore”: “CHANGE ME”,
      “vsphere-folder”: “CHANGE ME”,
      “vsphere-network”: “VM Network”,
      “vsphere-password”: “CHANGE ME”,
      “vsphere-server”: “CHANGE ME”,
      “vsphere-user”: “administrator@CHANGE ME”,
      “winadmin-password”: “CHANGE ME”
      }
      }

    Failures that occur with the Windows Autounattend.xml file

    • I’ve indicated the key edits you need to do within the related win 10 / server 2019 XML files
    • The main one that comes up, is the password, it needs to set in 3 places, shown below
    • The password provided in the script needs to be amended for your own environment, I would suggest using a password manager, or LAPS, but don’t leave the default !
    • Once you’ve chosen the updated password, amend it in the corresponding packer .JSON file as well

      <AutoLogon>
      <Password>
      <Value>20SuperSecure3d20</Value>
      <PlainText>true</PlainText>
      </Password>
      <LogonCount>2</LogonCount>
      <Username>Administrator</Username>
      <Enabled>true</Enabled>
      </AutoLogon>

      After </OOBE> section

      UserAccounts>
      <AdministratorPassword>
      <Value>20SuperSecure3d20</Value>
      <PlainText>true</PlainText>
      </AdministratorPassword>

      <LocalAccounts>
      <LocalAccount wcm:action=”add”>
      <Password>
      <Value>ChangeME</Value>
      <PlainText>true</PlainText>
      </Password>
      <Group>administrators</Group>
      <DisplayName>YourOtherAdminAccountHere</DisplayName>
      <Name>YourOtherAdminAccountHere</Name>
      <Description>Custom local admin</Description>
      </LocalAccount>
      </LocalAccounts>
      </UserAccounts>

    Failures that occur after the windows build has completed, but before Packer has finished it’s final steps

    • Here’s one I ran into this week. If you see the remote packer.exe process in the below state where it’s “waiting for WinRm to become available”

    vsphere-iso: Waiting for IP…
    vsphere-iso: IP address: 192.168.1.180
    vsphere-iso: Using winrm communicator to connect: 192.168.1.180
    vsphere-iso: Waiting for WinRM to become available…

    • This might be due to amending the vSphere-network from it’s default of “VM Network” to a VLAN that was filtering out various TCP ports
    • In the case I observed, once the VM was moved to the normal “VM Network” the issue was resolved.
    • So, you’ve got two choices, leave the build on the “VM Network” default shown below, or ensure that you’ve got the ports for WinRM open TCP 5985

      “variables”: {
      “os_iso_path”: “[CHANGE ME] CHANGE ME/WIN10_2004.ISO”,
      “vm-cpu-num”: “2”,
      “vm-disk-size”: “40960”,
      “vm-mem-size”: “4096”,
      “vm-name”: “WIN10-Packer”,
      “vsphere-cluster”: “CHANGE ME”,
      “vsphere-datacenter”: “CHANGE ME,
      “vsphere-datastore”: “CHANGE ME”,
      “vsphere-folder”: “CHANGE ME”,
      “vsphere-network”: “VM Network”,
      “vsphere-password”: “CHANGE ME”,
      “vsphere-server”: “CHANGE ME”,
      “vsphere-user”: “administrator@CHANGE ME”,
      “winadmin-password”: “CHANGE ME”
      }
      }

    Closing

    Please let me know if you found this blog post useful in the comments or on twitter

    The process was pulled together on the week of July 20th , 2020. I read over a bunch of blog posts and consulted the excellent forums on the packer.io site. To get it all working, much of the refined process came down to trial/error. I believe practical knowledge is key to success in our industry

    I’m really impressed with Packer thus far, the only “cost” was the time spent learning it, and will easily pay-out as I continue to use for build automation. Here, I’ve only covered using it with vSphere, but the packer folks support integration with dozens of the platforms, the sky is the limit

    Have a nice day!

    Maintaining a home lab for on-going career success

    Hi! I’m Owen, I work as a virtualization expert for a IT solutions provider in Quebec, Canada
    I’ve been working professionally in the IT industry since 2000, and from 2005 on-wards, have seen a huge benefit to maintaining a home lab
    The thing about working in the IT industry. Once you’ve completed the first part of education via a 1, 2, 4 year college or uni course, the learning doesn’t stop (or shouldn’t !). However, you probably won’t be going back to a traditional classroom setup for an extended period of time. It’s on you to pick-up distance ed course, follow online tutorials, read blogs, and try to implement as best you can via virtual labs.
    For me, though, the best way to learn something new is implement it on my home lab, and create a dependency on it, so when it stops working, I’m motivated to fix it, and by fixing it , I learn new things
    A recent example: I’d been interested in formally learning vmware for quite some time. In 2018, I finally took the plunge and enrolled in a distance education course from Stanly college on VMware VCP.  I completed the course over about 2 months using my nights, weekends

    I’ve had some form of home lab for years, but in 2019 I decided to get serious, and purchase “like” items to create a homogeneous environment for use as esxi hosts

    3x HP Elitedesk 800 G3 SFF w Intel Core i5 6500 CPUs
    And to each, I added the following:

    • 2x 16 GB RAM for 32 GB ram total
    • Samsung SSD 860 EVO 1TB SSD (vSAN capacity)
    • Samsung 970 EVO 1 TB NVM type SSD (vSAN caching)
    • Intel NIC quad card I350T4V2BLK
    • Intel X520 dual head 10 GB nic
    Connecting the 3 computers is a MikrotikCRS309-1G-8S+IN 8 port 10 GB switch
    With the hardware purchases/installs done, I installed esxi 6.7 U3 on each of them, and added them to my existing vCenter setup
    Next, it was time to setup vSan. I will tell you, I didn’t get this right the first time! But this is how we learn. I had originally tried a 2 node w external witness appliance setup, but found esxi host maintenance unsuitable, so switched instead to a 3 node N + 1 setup soon after.
    I finished my distance education course last July, but I’ve kept my vSan setup with no major changes, and it works great.
    It should be noted, that a full 3-node vSan cluster is not required for a home lab. In fact, you could do it for free via Hyper-V on your physical Win 10 laptop/desktop as long as you’ve got enough RAM/processing power. Else, VMware workstation is another alternative.
    Now! Why is the above my intro for this blog post? The investment of time/money has paid off, and I’ve started a new project as of April 2020 that’s related to VMware vSan for a new client with my current employer.
    Here are some other examples going back about 15 years. By luck or foresight , I’ve been able to identify something I didn’t know, implemented it in my home lab, put some services on it that I wanted to keep using, and ending up learning something new that I’ve been able to use in a production environment for a client/employer
    1. 2005 – Windows active directory
      I was enrolled in a college course that taught us MS Active directory, but the content was delivered at warp-speed, as it was mixed in with other courses. I was not retaining what was being taught, so setup my own AD @ home. The domain has gone through a few upgrades, but remains to this day! Knowing how to create a new AD setup from scratch is invaluable, and you never know when it will be required
    2. 2012 – Hyper-V
      For years, I was running Plex Media server on a physical host, but one day my partner at the time wanted to watch “i love lucy”, and Plex stopped working due to a recent config change. I decided to install / learn Hyper-V to virtualize my plex servers to enable easier snapshot/recovery, as well as setup TWO plex servers, so once could take over when the other was down for maintenance. I’ve still got this setup today, though the plex servers have been moved to vmware instead of Hyper-V
    3. 2015 – Microsoft RDS
      I setup Microsoft RDS for remote app usage to enable me to access published apps that I didn’t want to install locally on my various PCs. As part of setting this up, I learned about VHD based profile management, which served me well in 2019 when I needed to do an implementation with FSLogix!
    4. 2015 – Powershell
      I was previously familiar with regular windows .bat / .vbs scripting , but not Powershell. Towards the end of 2015, I learned powershell to enable me to automate disk clean-up on my file server. I’m still using these scripts today. I used what I learned at home to author 100’s of scripts for various contracts/projects over the past 5 years
    5. 2016 – Citrix XenDesktop 7.12 + StoreFront 3.8
    6. I was working for a client who was on the 7.6 LTSR track, but taking a course that was focused on features in the “current release” track. As such, I installed 7.12 at home, setup StoreFront and starting to learn the the differences between 7.6 and 7.12
    7. 2017Citrix Cloud
      After reading a bunch about Citrix Cloud on the social networks, I requested a trial to experiment with a hybird setup. I moved my Citrix DDC/SQL instances to a new Citrix Cloud intstance, and re-registered my existing VDAs to the cloud. I was impressed, the setup was short/simple. The trial ended, but learning the basics of Citrix Cloud helped me a lot in 2019 where I went into a client that was in the early POC for Citrix cloud.
    8. 2017 – VMware vCenter
      A lot of my career has been spent working for big financial companies, where they’ve got large vmware teams who had implemented vCenter before I’d started working there. I needed to do it for myself, so I signed up for the VMUG advantage program and setup my own vCenter via VCSA
    9. 2018 – Citrix ADC
      This is a common scenario: you’re working somewhere where all the initial build-outs are done. What do you do? Wait for one to break and require a re-build? no thx. I downloaded a trial of Citrix ADC VPX and went through a basic config to start using ADC at home
    10. 2018 – MDT
      Each place i’d worked previously, someone else was in charge of the creation / maintenance on the MDT task sequences, and they never seemed to work “quite right”. Can I do it better? yes I can. I took about a week to learn windows deployment service and MDT and still use it for new VM builds. Great for automation of builds, but I wouldn’t recommend it for app deployment, see point #10!
    11. 2018 – Microsoft DFS
      I set this up to cover fail-over events between my primary/secondary file servers. DFS can be great when setup correctly, but it’s often not. It’s good to know the ins/outs of this service, as it’s super common in the enterprise. Knowing it helped me a lot on a contract I was working on from July 2019-April 2020
    12. 2018 – Microsoft windows fail-over clustering
      I set this up as part of increasing my knowledge of SQL / SQL clustering. This knowledge came in super handy for SQL database migration work I was doing in 2018 , soon after I finished the work on my home lab
    13. 2019 – VMware vSan
      As i’m reasonable actively on twitter and receive a lot of newsletters, I’ve been reading about VMware vSan for years. In 2019, I decided to take the plunge and set it up at home, the details are in the first part of this blog post 🙂
    14. 2020 – PDQ Deploy
      My most recent implementation is to change the way I deploy software to my various virtual / physical assets at home. Previously , I was using chocolatey . It works ok for home use, but doesn’t really have that “enterprise” ready feel to it. PDQ Deploy offers a full GUI, so it’s an excellent choice for those environments where the local staff doesn’t have Powershell knowledge to maintain Chocolatey. I’ve not yet been able to use what I learned of PDQ Deploy with a client, but hope to soon! See my blog post on this topic , here
    I hope you found this blog post useful. The various examples of implementations on my home lab have served me very well over the years. I don’t have a crystal ball to know what’s coming next, but I do enjoy learning new things, and using my home lab means I can safely prepare for what might be next 🙂
    thanks for reading

    Home lab diagram summary for July 2020

    Greetings and welcome to the summer!

    Someone recently asked me why I had 5 vmware esxi hosts @ home. I realized some parts of my home lab aren’t documented , instead, their roles are trapped inside my head

    So, I put together a simple diagram of my current home lab
    I used the free draw.io to create it. It’s a handy online tool, that has a Chrome “desktop” app that I found performed better than the online version

     

     
     
    Create your website with WordPress.com
    Get started