Here’s a quick post in order to describe an issue I ran into after upgrading to MacOS Sequoia on an M1 Studio. After upgrade was complete I was prompted to enable the firewall (previously I used an additional product). Everything seemed fine and I was able to get on with using my system normally. I currently am running Sequoia 15.0.1 (24A348).
Around the same time I also upgraded to Citrix Workspace App 2409, especially because it introduced support for MacOS Sequoia https://docs.citrix.com/en-us/citrix-workspace-app-for-mac/whats-new#support-for-macos-15-sequoia.
At some point later that day I started to experience disconnection from Citrix sessions upon the 30 and 60 minute session lifetime, which incorrectly sent me down the route of checking timeout settings firstly and then other suspicious network observations which turned out to be unrelated.
What follows to my regret was starting a lengthy sequence of troubleshooting steps including Wireshark, building Azure VMs in different geo-locations, downgrade of Citrix ADC VPX Gateway, testing on Windows VM in my Mac, throwing the kitchen sink at it in fact.
Only when I began investigating further – by building a Windows 11 VM in the geographical location (in order to eliminate inter-country internet links) – did I find that the Citrix session within the desktop was rock solid. Instead, the problem shifted to my MacOS based RDP connection to the said temporary desktop, along with error 0x407 “Your session ended because of a data encryption error”.
At which point I realised that the session resets occurring periodically under ICA were also a symptom of the underlying session terminating, in a similar way to the RDP protocol.
Searching more widely I came across a few articles, linked below (with thanks) and discovered that disabling the Mac firewall then immediately resolved the problem with both RDP and ICA connections. I upgraded the MS Remote Desktop app with the new version (now called Windows App) and retested my session stability. I checked my Citrix Workspace App for Mac was updated fully and everything is now working fine, but only with the firewall switched off and a third-party app doing its job instead.
In summary – there appears to be a problem currently with the firewall implementation in Sequoia, and I look forward to seeing this patched soon.
https://discussions.apple.com/thread/255799644
https://www.reddit.com/r/MacOS/comments/1fizxc9/ms_rdp_broken_on_macos_sequoia/
https://www.theregister.com/2024/09/23/security_in_brief
Maintaining long lived Tanzu clusters
Very few resources provide real guidance on what to do after creating a cluster using Tanzu TKG, particularly in terms of ongoing maintenance beyond the initial handoff to a developer.
What often happens next is you eventually learn of a problem once the system has long since become stable and adopted for general use, and this puts you straight away on the back-foot in terms of overcoming the issue.
This post concentrates on the kinds of problems you might run into during operational management of a cluster, however it doesn’t claim to capture every such problem, just those which I’ve personally been involved in troubleshooting.
Of course – the VMware documentation should be your go-to place on the first occasion, so do check through the known issues of the release notes for your specific version first before continuing with any other activity.
You can find a link to the general product documentation for Tanzu TKG in the final section, which include the most recent versions by default, and even an archive for older versions.
Do cluster credentials even expire?
Well, this is something which we’re unfortunately going to have to learn, most likely the hard way, and to my mind this is not sufficiently sign-posted within VMware TKG documentation.
Here’s a description of this scenario and the task to update your credentials, using the TKG 2.1 documentation as an example: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/2.1/tkg-deploy-mc/mgmt-manage-index.html
This approach assumes that you can still access the cluster using kubectl commands in order to retrieve the current kubeconfig data of your CLUSTER-NAME:
kubectl -n tkg-system get secrets CLUSTER-NAME-kubeconfig -o 'go-template={{ index .data "value"}}' | base64 -d > mc_kubeconfig.yaml
Once you have obtained this data you’ll want to know when the credentials expire, so use the following method to decode just the client certificate element first and then use openssl to extract the date elements:
kubectl -n tkg-system get secrets CLUSTER-NAME-kubeconfig -o 'go-template={{ index .data "value"}}' | base64 -d | grep client-certificate-data | awk '{print $2}' | base64 -d | openssl x509 -noout -dates
notBefore=Aug 21 15:26:11 2023 GMT
notAfter=Feb 19 03:31:32 2025 GMT
Compare these dates (obtained from the cluster) to what you have stored locally (held within your current kubeconfig context) using:
kubectl config view --raw --minify | grep client-certificate-data | awk '{print $2}' | base64 -d | openssl x509 -noout -dates
The output should be the same, but if it is not then you can update your local kubeconfig file copy of the cluster’s data using the mc_kubeconfig.yaml file outputted earlier.
Now is a good time to make a date in the diary to either upgrade your Tanzu TKG implementation or manually rotate the certificates before this date arrives. Please refer to the general guidance here kb.vmware.com concerning rotation.
Thankfully this issue has been resolved in TKG 2.1.x via the auto-renew feature which can be retrospectively changed by editing the cluster object:
Misplacing the keys to the castle
Losing cluster admin credentials
Individual client certificates stored within Kubeconfig files generally expire after 6 months, and the kubeadm generated certs (seen below) automatically expire within 365 days of the cluster being built. Only the three certificate-authority certs created within the Kubernetes cluster last 10 years by default.
Here is the output generated from a control-plane node using the command:
kubeadm alpha certs check-expiration
This shows the output from a cluster created a few minutes ago, hence <365d residual time remaining.
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Mar 22, 2025 10:11 UTC 364d ca no
apiserver Mar 22, 2025 10:11 UTC 364d ca no
apiserver-etcd-client Mar 22, 2025 10:11 UTC 364d etcd-ca no
apiserver-kubelet-client Mar 22, 2025 10:11 UTC 364d ca no
controller-manager.conf Mar 22, 2025 10:11 UTC 364d ca no
etcd-healthcheck-client Mar 22, 2025 10:11 UTC 364d etcd-ca no
etcd-peer Mar 22, 2025 10:11 UTC 364d etcd-ca no
etcd-server Mar 22, 2025 10:11 UTC 364d etcd-ca no
front-proxy-client Mar 22, 2025 10:11 UTC 364d front-proxy-ca no
scheduler.conf Mar 22, 2025 10:11 UTC 364d ca no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Mar 25, 2033 11:44 UTC 9y no
etcd-ca Mar 25, 2033 11:44 UTC 9y no
front-proxy-ca Mar 25, 2033 11:44 UTC 9y no
Upcoming kubeadm certs expiry
If you are now approaching the expiry date of the kubeadm certificates and have not automatically renewed these certs, do so by either completing an upgrade to a newer Tanzu version, or by control plane scaling in which case they can be automatically rotated whilst remaining in the current state.
Manual rotation process is described here: https://kb.vmware.com/s/article/86251 – BUT crucially this requires SSH access to at least one of your control plane nodes. See the section below on losing access for some possible recovery options.
Once you have rotated the certificates on the control-plane don’t forget to also refresh the content of the following two sections,
client-certificate-data: [updated data from admin.conf]
client-key-data: [updated data from admin.conf]
in BOTH the files of your tanzu CLI machine below: (used for kubectl workload contexts and tanzu login management contexts respectively)
~/.kube/config
~/.kube-tkg/config
After the management console side of things are refreshed correctly you can then have confidence of another year’s administrative access. Now move on to rotate the certificates on any worker clusters which that management cluster also oversees using the same process (but you don’t need to update the tanzu CLI version of the kubeconfig file for worker clusters since you don’t log in to them).
Retrieve admin credentials before expiry
Approaching the time of your individual kubeconfig client-certificate expiry you simply need to retrieve a new file using tanzu cluster kubeconfig get clustername --admin --export-file new_file_name
However, if you don’t refresh your admin credentials periodically then you may eventually find that after 365 days of operating a stable cluster that you no-longer have access to it at all via kubectl or tanzu login commands.
In situations where your computers are kept isolated from the internet, you might run into a problem where, without prior notice, you can’t use your kubeconfig file to talk to your cluster. This is especially true if you’re not updating your system more than once a year.
It is recommended to maintain awareness of the cluster’s certificate expiry dates and complete rotation beforehand, including to refresh your kubeconfig file via tanzu CLI. If you are told about a cluster’s access expiring after the fact but if you still have SSH access then all is not lost. You should then connect to the cluster control-plane and carry out manual rotation then retrieve the updated content from the /etc/kubernetes/admin.conf file and place the data into your local kubeconfig file.
Losing SSH access
What if you lose access via SSH? In the TKG 1.6.x release a security hardening issue https://kb.vmware.com/s/article/90368 can cause attempts to logon via capv@controlplaneIPaddress to fail, requiring additional edits to the cluster’s kcp and kubeadmtemplate before you will be able to log in.
An alternative approach might be to scale your control plane nodes using vertical scaling, e.g. by modifying the size of a node’s attached hard disk, CPU or memory spec for instance. This process is described here: https://kb.vmware.com/s/article/91164. By scaling your control plane new VMs will be spun up, each with the possibility of regaining access via 60 day SSH access (Ubuntu) or 90 days (Photon). This security feature can be disabled (see Method 2 in the referenced document) but only once you have regained connectivity.
More desperate measures might be required in the event that both your kubeconfig and tanzu login access is no longer possible. I do not confirm nor recommend the manual removal (via vCenter) of a control-plane VM – in the event that you lose both SSH and tanzu CLI access, but my suspicion is that if there are more than one control-plane nodes the KubeadmControlPlane will no longer match the running spec and a new node will be provisioned, along with SSH access reinstated. This is something I aim to test further.
Contour package certificate expiry
VMware’s documentation for installing the Contour package into a worker cluster is rather generic, but one of the ways in which you can extend your workloads is by installing the Contour/Envoy ingress controller along with some default values. This scenario is described as a CLI managed package, beyond which the basic process for installation is detailed below (for TKG 1.6):
tanzu package available list contour.tanzu.vmware.com
tanzu package available get contour.tanzu.vmware.com/1.20.2+vmware.1-tkg.1 --generate-default-values-file
Within the generated default values file is a snippet which defines a TLS certificate lifetime which is consumed by Envoy and Contour pods when communicating with each other over gRPC protocol.
certificates:
duration: 8760h
renewBefore: 360h
The first value defines the period after which these internal-use Contour certificates will expire, and the renewal period before which they should be updated. However this can cause a strange problem with Envoy pods if you installed Contour a couple of days after spinning up a new cluster – you will see something like:
StreamListeners gRPC config stream closed: 14, upstream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: TLS error: 268436498:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE
https://kb.vmware.com/s/article/90811 details the solution, which is simply to remove the secrets and have the package automatically recreate them. I have also deinstalled the package using the tanzu CLI and reinstalled it without any other problems, however you should be aware that your cluster’s certificates might expire before Contour has recreated the certificates.
List of resources and useful pages
https://docs.vmware.com/en/VMware-Tanzu/index.html
https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/index.html
https://docs.vmware.com/en/VMware-Tanzu-Packages/2024.2.1/tanzu-packages/ref.html
https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/services/tkg-doc-archive-2x.zip – this is a ZIP file archive of the TKG 2.x version documentation
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs
Avi NSX licensing resources
https://avinetworks.com/docs/latest/nsx-alb-license-editions
https://docs.vmware.com/en/VMware-NSX-Advanced-Load-Balancer/22.1/Administration_Guide/GUID-B5EC8F3B-A75E-4809-A653-6EBE08CFED81.html – Avi licensing
Comparing YAML documents with Beyond Compare
I’m a huge fan of Scooter Software’s magnificent Beyond Compare tool, having used it for years in many different scenarios. It does a brilliant job of lining up the similarities and differences in multiple file types – especially text based file formats.
A Kubernetes container orchestration project likely involves defining multiple YAML manifests for creating different objects in the API so I often spend many happy hours comparing differences between folders full of files in different environments.
Unfortunately within Beyond Compare there isn’t any inbuilt YAML file support so it sometimes makes mistakes when aligning functionally equivalent but structurally different files.
Here’s an example:
By default red text shows differences, blue text means minor differences and hatched areas show where there is a lack of alignment possible. I have defined certain non-important grammar elements to ignore, e.g. secretName: so these are shown in blue, but why is it not possible to line up the other elements?
In discussion with Scooter Software they explained that they would need a YAML parsing algorithm to be developed in order to pre-sort such lists into the correct layout, and this isn’t possible right now.
Alternatively they suggested creating a custom File Type which uses an external program to do the sorting work before it attempts to line up and search for differences.
With that in mind I came across the yq project in Github by Mike Farah which provides amongst other things a sorting tool along the lines of jq (for JSON) for YAML documents instead. The following brief instructions show how to create a custom file type in Beyond Compare to have yq sort the keys and values in a temporary file before showing the comparison view. You can then choose to integrate and save changes on either side.
Follow preparation steps
- Download
yq_windows_amd64.exefrom the release page and rename the file tojq.exefor ease of use - Create a folder called
YAMLin%USERPROFILE%\AppData\Roaming\Scooter Software\Beyond Compare 4\Helpers - Copy the
jq.exefile into the above folder - Create a new file within the same folder
yaml_sorted.batcontaining:@echo off
Helpers\YAML\yq.exe e -M "sort_keys(…)" %1 >>%2 - Open Beyond Compare and configure as follows
Configure Beyond Compare
Choose Tools, File Formats
Choose the [+] button and select Text Format
On the General tab enter a file mask to use when applying the File Format correctly, e.g. *.yaml,*.yml
Click on the Conversion tab and select External program (Unicode filenames) then enter the following path and parameters
Helpers\YAML\yaml_sorted.bat %s %t
Optionally, choose whether to Disable editing (when displaying the resulting comparison) and whether to ‘Trim trailing whitespace’ when saving – I find this helpful sometimes.
Click on the Misc tab and choose ‘Insert spaces instead of tabs’ and tab stop 8.
Click Save and Close.
Now find two YAML files to compare and double click in Beyond Compare. It will use the file type mask to apply the custom file format you just created.
In the background it will run the yq executable twice (via the batch file), once for each input file (left and right comparison). The executable takes the input file (%1) then sorts the keys and values alphabetically in sequence first before outputting the content to a new temporary file (%2).
Now open a YAML file comparison and you’ll find a much improved alignment based upon first the sorting of individual keys, then subsequent scalar values within each.
e.g. yq e -M -P "keys" C:\Users\stwa\desktop\example-ingress1.yaml
Here is the resulting comparison after processing with yq. I think you’ll agree that it’s much easier to spot the differences and also allow standardisation of your files going forward.
There’s much more that you can do with both yq and Beyond Compare to further tune the formatting order, but hopefully this gives you some sort of light at the end of the tunnel when attempting comparisons of YAML documents.
I would like to investigate whether it’s possible to define a template format in yq so that it uses a custom key alignment order so that it matches the specification used by the Kubernetes API. In addition Beyond Compare allows line weighting within the algorithm so that it can assign preferential weights to certain grammar elements, although this has not been particularly fruitful when used according to the example below.
Final thoughts
Custom file types don’t just affect comparisons – If you right click on a file and choose Open With, Text Edit you will also see the sorted version of the file – not the original.
NB – If you save the modified content it will be saved using the new sorted format.
Documentation for yq is available here: https://mikefarah.gitbook.io/yq
Citrix XenApp/Desktop LTSR 7.15 Azure catalog creation issues
I came across this problem whilst trying to build a lab scenario with an older version of LTSR 7.15 and wasn’t able to find any similar issues documented elsewhere. Essentially Citrix Studio would not allow me to browse for .vhd files when creating a new catalog from an unmanaged disk located in an Azure storage account.
Here’s the troubleshooting process and solution at the end (spoiler – it’s TLS 1.1, 1.2!)
Trying to create a catalog following successful creation of a hosting connection:
Machine creation wizard error
You might find for instance when examining other storage accounts that you are even able to view the name of any named containers e.g. ‘logs’ located within the storage account object, but no obvious difference is possible.
You might try even using PowerShell to examine the hypervisor connection, and by following along will eventually reach a dead end in the communication with Azure:
Add-PSsnapin Ci* cd XDHyp:\ cd HostingUnits (dir).PSChildName
Determine the name of your hosting connection, and change directory into it
cd .\YourHostingUnitName\ (dir).PSChildName
Determine the name of your resource, and change directory into it
cd .\image.folder\ (dir).PSChildName
Determine the name of your Azure resource group, and change directory into it
cd .\YourResourceGroupName.resourcegroup\ (dir).PSChildName
Determine the name of your storage account, and change directory into it
cd .\YourStorageAccountName.storageaccount\ (dir).PSChildName
At this point if you attempt to use dir or get-childitem you will receive an error saying:
An exception occurred. The associated message was Error: Could not receive inventory contents from path
In summary you don’t receive very much information from Citrix Studio which might provide further assistance at troubleshooting the issue. Citrix Host Service will generate an Event ID 1007 message including the text:
Citrix.MachineCreationAPI.MachineCreationException: Error: Could not retrieve inventory contents from path /UK South.region/image.folder/YourResourceGroup.resourcegroup/YourStorageAccount.storageaccount ---> Microsoft.WindowsAzure.Storage.StorageException: The remote server returned an error: (400) Bad Request. ---> System.Net.WebException: The remote server returned an error: (400) Bad Request. at System.Net.HttpWebRequest.GetResponse()
The solution took quite some comparison between different working environments until I happened on the cause and eventual solution. That is, that the storage accounts affected were configured by default to use TLS 1.2 as a minimum rather than TLS 1.0. Clearly this isn’t ideal but even relatively recent LTSR 7.15 CU5 (and presumably earlier) does not seem to support TLS 1.2 for this type of API communication with Azure.
Simply locate the storage account and modify the following switch under the Configuration page:
Finally (after waiting 30 seconds or so for the storage account change to take affect you’ll be able to open the storage account and view the unmanaged disk VHD blob.
Switching to TLS 1.1 support does not improve the situation, it will begin failing again – even though the browser in Windows Server 2016 (with recent updates) supports TLS 1.1 and 1.2. So it appears that the code somewhere is out of date in LTSR 7.15 (either Citrix Studio or PowerShell perhaps).
I’ll update this post if I manage to resolve it using another method, but in my experience after testing this problem goes away with LTSR 1912.
Intel NUC 7th Generation with Thunderbolt Ethernet on ESXi 7
My 3 host ESXi 6.5 home lab was built a couple of years ago now in order to develop some vSAN test scenarios that I was assisting a client with. Now that lab is in the process of being repurposed to become an NSX-T / vSAN lab running ESXi 7.0.
My three NUCs are based on the 7th Generation Intel NUC7i5BNH with 32GB RAM, onboard M2 NVMe SSD cache disk (256GB) and a 512GB SSD capacity disk.
– By the way, in case you’re wondering – it flies, even over 1Gbit ethernet with all-flash vSAN and deduplication/compression turned on. However – don’t use something like this for production as you’ll need to be prepared to lose ~14GB RAM per host for the dedupe in-memory object map and all-flash requires 10Gbit ethernet!
However, back to the issue in hand. One of the first issues I ran in to when assessing the requirements for NSX-T in a fully collapsed cluster (running vSAN, vCenter and NSX Manager) was a need to have two physical network interfaces (pNICs) but my Intel NUC 7th Gen hosts only have a single on-board gigabit ethernet adapter. This isn’t front page news of course as William Lam has been documenting the use of USB based ethernet in lab scenarios for quite some time, originally resulting in a VMware Fling he coauthored with another device driver engineer (Songtao Zheng) at VMware.
Now I began to question with recent releases such as VMware ESXi 7.0 whether any of the drivers or settings mentioned would even be required in order to get a second ethernet adapter working. This post is really just a signpost for people who might be doing similar things in their own labs.
I decided to jump in with both feet by purchasing three Startech USB-C 1-Gbit Ethernet adapters from Amazon UK. These devices use the Realtek RTL8153 chipset. My NUC devices have Thunderbolt 3 interfaces (with the lightning bolt marking) but I was pretty sure that the USB-C connector would work as the interface supports USB 3.1 Gen2 devices.
UPDATE YOUR BIOS – mine hadn’t been touched since 2017 so I went online first and downloaded the latest update for my NUCs, then flashed using a USB key with the .bio file and F7 key during boot.
UPDATE YOUR ESXi release – reading one of William Lam’s posts I found that there was a recent patch release of ESXi 7.0.0b which includes an updated USB driver:
VMware-vmkusb_0.1-1vmw.700.1.25.16324942
As far as I can tell this includes the ability to detect Thunderbolt connected devices amongst other improvements, but this awareness certainly negates the need to disable any existing VMware USB driver (which older posts I’d read had discussed prior to installing the USB ethernet fling).
INSTALL the VMware ESXi 7.0.0 release of the USB Fling:
esxcli software vib install -d '/vmfs/volumes/QNAP_VMFS_DS01/tmp/ESXi700-VMKUSB-NIC-FLING-34491022-component-15873236.zip'
USB Native Driver Fling for VMware ESXi | 0.1-4vmw.700.1.0.34491022
3. Shut down the ESXi host, you’ll need to go into the BIOS at next boot
ENABLE THUNDERBOLT BOOT in BIOS -until you do this you won’t be able to see any USB 3.x network devices. William Lam again has the lead again, with this linked post concerning Thunderbolt 10Gbit adapters on Intel Skull Canyon devices. Enter the BIOS and enable THUNDERBOLT BOOT.
Before enabling this feature you’ll find that 'lsusb -tv' will only show a single USB XHCI root HUB:
4. Save your BIOS settings, connect the Startech USB device and boot ESXi. Once reloaded compare the ‘lsusb -tv’ result with the previous version.
NB – If you find that your USB adapters are only connected at 100Mbit/s then it’s likely that the default ESXi 7.0.0 drivers have been loaded instead of the ones provided in the Fling. You’ll also see that the adapter name is detected as ‘cdce’ instead of ‘uether’. In this case make sure that the drivers are installed correctly and try a reboot with the adapter connected.
[OPTIONAL] if you have any 10 Gbit Thunderbolt adapters you could also use the following steps to add the Marvell drivers. I haven’t actually acquired any of these yet, but the instructions should be good as I’ve tested the installation process itself.
INSTALL the VMware release of the Marvell Atlantic USB driver:
- Download the .zip file and upload it to a datastore that your hosts can access.. e.g. /vmfs/volumes/58134191-c9bf8fe8-d464-d067e5e666da/tmp/MRVL-Atlantic-Driver-Bundle_1.0.2.0-1OEM.670.0.0.8169922-offline-bundle-16081713.zip
- Enter maintenance mode and vacate the ESXi host
- Install the offline bundle VIB using:
esxcli software component apply -d /vmfs/volumes/QNAP_VMFS_DS01/tmp/MRVL-Atlantic-Driver-Bundle_1.0.2.0-1OEM.670.0.0.8169922-offline-bundle-16081713.zip
Native atlantic network driver for VMware ESXi | 1.0.2.0-1OEM.670.0.0.8169922
PowerCLI Get-Tag fails with ‘Could not load file or assembly ‘Newtonsoft.Json, Version=10.0.0.0’
Here’s a simple scenario which I came across today. You would like to work with your vSphere environment using the latest PowerCLI but discover that v6.5.1 is the latest downloadable version on VMware’s website. Hearing that the distribution for this code has now moved to the PowerShell Gallery you open a PS prompt and enter:
PS:\> Install-Module VMware.PowerCLI
The modules are downloaded and installed successfully, and you are able to connect to your vCenter environment:
Connect-VIServer -server vcenterserver.com -user 'DOMAIN\username'
But when you attempt to use a simple command such as:
Get-Tag
you receive an error similar to:
get-tag : 11/10/2018 21:06:20 Get-Tag Could not load file or assembly 'Newtonsoft.Json, Version=10.0.0.0,
Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file
specified.
At line:1 char:1
In my case I found that other system components on my VM were using an older version of the Newtonsoft.Json.dll (e.g. Citrix Virtual Desktop Agent) that were found in the file search path before the PowerShell module’s location.
Searching for the file conflict using ProcMon I noticed that the Connect-VIServer cmdlet does indeed find and load a version of this .dll during the connection process, e.g. the one located in:
C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Newtonsoft.Json\v4.0_4.5.0.0__30ad4fe6b2a6aeed\Newtonsoft.Json.dll
However this version is 5.0.5.16108 on my Windows Server 2016 platform and we’re looking for 10.0.0.0 or newer.
Work-around
Retrieve the newer version of the file (supplied with the PowerCLI modules), located for instance in:
C:\Users\username\Documents\WindowsPowerShell\Modules\VMware.VimAutomation.Common\net45
and place a copy in somewhere PowerShell is likely to find it, e.g.:
C:\Windows\System32\WindowsPowerShell\v1.0\Newtonsoft.Json.dll
This simple work-around proved successful for me, but you should check of course to verify all other functionality which might depend on this file before making a similar change in a production environment.
Bespoke consulting for Virtualisation and Server Based Computing environments
SBC PureConsult is a specialist IT consultancy dedicated to the design and implementation of Citrix XenApp/XenDesktop, VMware vSphere and vCloud based projects. Over the last 18 years we have carried out numerous projects in various industry sectors, and in several cases pioneered custom solutions for implementing other vendor’s software upon a virtualised application delivery platform.
We have enjoyed significant success in the Hospitality and Leisure markets and defined the standard for delivering Micros Fidelio’s Opera PMS/ORS product using Citrix XenApp/Presentation Server. In fact, we have successfully completed PMS deployment projects for a number of top-tier hotel companies including delivery of desktop and application environments for several hotels using a private cloud model.
We have also developed specialist knowledge in the Communications Regulations sector surrounding the best practice implementation of LStelcom AG SPECTRA series of applications in a Citrix XenApp environment. Following success gained in deploying the SPECTRA solution using Citrix XenApp this has become a core strength within our portfolio.
We strongly believe that whilst you can read about our values and core competencies here, talking to us about your specific needs and expectations should be just as rewarding. Please take the time to read through the short introductions to our business and should you have any questions or queries don’t hesitate to discuss them with us.
Click here to view our proposition concerning the implementation of Micros Systems, Inc. Opera using Citrix.
Please contact us to hear about our specialist knowledge working with the LStelcom AG, SPECTRA suite.
Office 365 for Mac, Outlook unread count wrong
I recently received a new MacBook Pro and restored all of my previous applications and data from a Time Travel backup. One small issue that I noticed afterwards was that the Unread mail count (1) was incorrect, since even when I set a Filter to show only unread items there were no remaining mails shown. Despite a quick search for the answer online it seems that Office 365 (Outlook 15.0) for Mac is not widely written about yet. The solution I fell upon was quite simple (please be careful to check that your mailbox is correctly synchronised before beginning):
- Select the folder which shows the incorrect item count.
- Choose Properties on the folder.
- Click Empty Cache, in order to remove the local copies of the mailbox folder items (this assumes you’re using the Exchange mailbox as a primary store and not a POP server etc)
All mail items were then immediately removed from the local mailbox cache, following which you can right click on the folder concerned and then choose Synchronise Now.
This simple fix easily resolved my problem.
Purple screen halt on ESXi 5.5 with Windows Server 2012 R2
Believe it or not, but it seems that it is possible to crash a clean ESXi 5.5 host right out of the box by installing a Windows Server 2012 R2 virtual machine with an E1000 virtual network adapter and attempting a file copy to another VM located on the same box.
I was trying recently to copy some data from a Windows Server 2003 VM onto a new 2012 R2 VM on the same host. Expecting that the file copy should be extremely fast (due to proximity of network traffic on the same switch) I was left scratching my head when I noticed only 3-10MB/s transfer rate.
Because I was still running ESXi5.0 I thought it would be better to troubleshoot if I upgraded to the latest version of the hypervisor, only to find that the second I hit ‘paste’ to begin the file transfer the entire hypervisor crashed with a purple screen.
Needless to say, this isn’t a fringe case and others would appear to have noticed this behaviour too. The fix is simple enough, just swap out your E1000 vNIC on the 2012 R2 server with a vmxnet3 adaptor, but how is this simple scenario so dangerous that it is able to take out a whole host?
Thankfully, after swapping the vNIC I was then able to achieve 50-60MB/s throughput continuously, which was more than enough of an improvement given where I started before.
I’m going to link to the original post I found here, but nevertheless I’ll update this page if I find that there is a known issue somewhere that explains how this behaviour has occurred.
Dell MEM, EqualLogic and VMware ESXi, how many iSCSI connections?
I’ve been working on a fairly large cluster recently which has access to a large number of LUNs. All 16 hosts can see all of the available disks, and so the EqualLogic firmware limits have started to present themselves, causing a few datastore disconnections. As part of the research into the issue I came across several helpful documents, which hopefully should prove essential reading in case you haven’t come across the planning side of this before:
A description of Dell MEM parameters, taken from EqualLogic magazine
Dell EqualLogic PS Arrays – Scalability and Growth in Virtual Environments
EqualLogic iSCSI Volume Connection Count … – Dell Community
Best Practices when implementing VMware vSphere in a Dell …
Configuring and installing Dell MEM for EqualLogic PS series SANs on VMware
If you run into problems with iSCSI connection count then you will need to rethink which hosts are connecting and how many connections they maintain.
These factors are detailed within the documents linked to above, but in brief, you can attempt to resolve the issue by:
- Reducing number of LUNs by increasing datastore sizes
- Reduce the number of parallel connections to a LUN that MEM initiates
- Use access control lists to create sub-cluster groups of VMs that can see fewer LUNs
- Break your clusters down further in order to separate different groups of disk from each other, e.g. on a per-storage-pool cluster basis
