Category Archives: Windows

Windows with WSL2, a good configuration for a dev team?

In the Talentoday tech team we have regular discussions about the best Operating System (OS) workstation configuration for the team. While experienced engineers are free to choose what they prefer, it is reasonable to recommend and maintain a kind of preferred configuration for interns and juniors to facilitate the setup and onboarding. Talentoday tech stack being *Nix based, for long most of the developers were using MacOS with MacBooks. This is definitely an acceptable solution but we confronted that, and some of us (me included), started to use Linux distributions. However, I did not find yet an ideal setup. I will share here some criticisms about various OS configurations and discuss the new Microsoft WSL2 and see how this could be the best of both all worlds, yet to be confirmed.

Usual OS configurations, main flaws

First I am presenting here my main criticisms, as you will notice most are not on the technologies themselves but more on their daily usage within a team and a potential large fleet of devices.

Windows OS. Unless you develop solely .NET applications (the regular .NET FX not the new .NET Core) that can only run on Windows OS then, Windows is probably not the ideal dev OS. While Python interpreters work well on Windows some other technologies, like Ruby for instance, are really a pain on Windows. It is a fact that most of developer tools and technologies are not thought for Windows (or at least not as a first class citizens). On the positive side, administrating the team machines would be easier compared to others. Indeed, Windows is really adapted for maintaining a large fleet of machines and most everyday applications run perfectly on Windows. Unfortunately, this latter argument does not counterbalance the poor dev experience on many technologies.

Linux standalone OS setup. In this case, you have a real Linux OS that will probably be the closest thing that hosts and runs your applications in production. Using Linux, you can use a great package manager such as apt. Good code editors like VSCode or Sublime work perfectly. Yet… in the real world other problems arise. First, the ones that are due to Linux desktops and window manager themselves. For example, having different resolutions and scaling for your screens is not well supported unless you go with Weyland. Well with Weyland you will stumble on troubles for sharing your screens. In our team, we do remote work, we need to share things when huddling. We have to present to clients and teammates around the world everyday. You need a comfortable setup with multiple monitor and the ability to jump on in case clients invite you on a GoToMeeting, Zoom.us or whatever videoconf system and this needs to work. Similarly, it happens frequently that we receive *.xlsx, *.docx files with macros that only Office desktop can process. Therefore, even if you manage to have everything working on your Linux desktop workstation there are things, independent of your will, that can sometimes be problematic.

Dual boots. Honestly, I never was really convinced by dual boots. It is quite cumbersome. In addition, most of the time you need to disable secure boot, which is, for a company setup, probably not a good recommendation.

Leveraging virtualized OS. That’s an option and here are two possibilities. First, host OS is Windows/MacOS and you develop in a guest Linux. Well if you are a developer your primary work will probably be development, then, even though you can make your VM as seamless as possible, it is always preferable to do your everyday work in the host OS. The other way is setting a guest Windows OS (you cannot guest MacOS) on Linux host. Again, some problems like sharing your Linux screen for showing your local devwork will not go away. Let me also point out that you depend a lot on the hypervisor and its ability to handle well displays. My experience proved me that this is often bugged. Fixing this kind of bugs does not seem to be the top priority of hypervisors. Take for example Oracle and VirtualBox that keeps ignoring for years this bug on HighDPI screen.

Mac OS. It seems that this has become a very popular solution for developers in the past decade. This is the configuration that I have now for almost two years and it works!  Nevertheless, there are some criticisms that cannot be ignored. Even if MacOS is Unix based and POSIX compliant this is no Linux. This makes a bit of difference: you cannot use apt-get but you have to rely on Homebrew instead. Small stuff that make your local config different from the prod systems. One can not ignore that Docker For Mac also has some serious flaws. In addition, it is clear that maintaining a fleet of Macbook Pros is quite a pain: no native group policies . For device maintenance, you have to go with Apple support etc. Finally, the cost associated with purchasing Macbook Pro compared to its equivalent at Dell is higher (even that is not so much as it is often mentioned in discussions).

The best of all worlds? Windows 10 with WSL2?

Ubuntu on Windows

Ubuntu on Windows

WSL stands for Windows Subsystem for Linux. At the time of  writing its first version WSL1 is shipped on all up-to-date Windows 10 builds, to get access to WSL2 you need to register to Windows Insider Program and retrieve a preview build.

With WSL, Microsoft brought a Linux Kernel inside Windows 10. It can be used directly from within Windows host using a bash shell. In its first version, it was really useful for many small tasks, such as sshing easily from Windows (and not relying on PuTTy). Yet, some limitations on networking and filesystem did not make it a viable solution for an everyday developer workstation setup. WSL2 is now a much more complete solution that makes usable for development (involving a full VM on top of Hyper-V that some may even see as a regression).

In addition, some really important news came along. First, the popular code editor VSCode fully supports WSL2 and brings extensions. Second, and probably more important, Docker in the latest tech preview edition of Docker Desktop let you use WSL2 to run the containers.

If you are working on existing projects, there is a strong possibility that some (if not all) of your services leverage Docker. Therefore, you must have a way to run and access Docker containers from your new WSL2 environment. This was our situation at Talentoday where we rely a lot on Docker.

I tried to install our *Nix based Talentoday stack on Windows 10 with WSL2 and it worked like a charm.

As a big overview, at Talentoday, the applicative server services are built on top of Ruby on Rails (with jobs and queues etc.) accessing SolR indexers. We also have a cache mechanism on top of Redis, a PostgreSQL database and a Flask based Python WebAPI. For the developer experience, we rely a lot on Docker containers (including the database) for this various components. With the Docker Tech Preview, I had no problem building and starting my images and contacting them within WSL2 and accessing them from both WSL2 code and/or host Windows.

We will need some step back and see if WSL2 is the right way to go in the long run. We will have to pursue our investigation and see if there are no other problems (performance, networks etc.) that I could not spot during this one day trial. In all cases, this early attempt sounds extremely promising and could be a good solution for developers in teams who want to keep the benefit of a commonly distributed OS and a real developer experience leveraging *Nix based technologies.

Talentoday stack running on WSL2

Talentoday stack running on WSL2

A sample Wix installer using the ActiveSetup feature

Windows Users

All users have the software in the case of a local machine install contrary to user install

In this post I will explain the technical details of the Wix sample installer for local machine install of Excel-DNA addins. Wix is a popular free toolset to create .msi installer. I used the ActiveSetup feature to address a “per user” install problem and this is the part I will cover in this post. Therefore, it maybe useful for both who do not know ActiveSetup at all and for people looking out to write their own Wix installer leveraging the ActiveSetup. Even, if I think ActiveSetup should be used only if there is no alternative this barely known feature may save your life so it is good to be aware of its existence. We will use the terminology of the Windows Registry that you can check on the Wikipedia page.

First let us present the problem and why we needed this ActiveSetup feature. Automation addins and by consequent Excel-DNA ones are registered by setting a value OPEN in the current user registry hive (HKCU). This OPEN value contains the path to the packaged Excel addin which a file with .xll extension. If there are two addins to register then the values should be named OPEN, OPEN2, OPEN3… see this StackOverflow discussion or this MS documentation. This is a per user registration, if you create a new user, then he will not have the addin starting with Excel. There is a WixSample available on the Excel-DNA github. This is fine as long as you need only a per user install, however, for enterprise deployment you need most of the time a local machine install. Here starts the troubles: the OPEN value mentioned above cannot be set as a HKLM (machine) subkey only in the current user hive (HKCU). Therefore we have to find a way to setup this OPEN value for all users and this is where the the ActiveSetup comes into play. We have also tried to set the addin in the XLSTART machine directory but this has also drawbacks see this discussion

So what is this ActiveSetup feature? ActiveSetup is a simple yet powerful mechanism built-in in Windows that allows you to execute a custom action at logon for new user. How does it work? Actually this is quite simple. You have to create a subkey in the HKLM hive Software/Microsoft/ActiveSetup/Installed Components. Typically this subkey is a GUID that contains a StubPath REG_EXPAND_SZ value whose data points to the executable of your choice. This value is mirrored in the HKCU registry. Precisely, at logon windows checks if you have this ActiveSetup key in the current user HKCU hive, and if not, the executable of the StubPath is invoked. If the key is present in the HKCU registry then nothing happens. The nice thing is that localization, version and uninstall are handled. Let us detail the uninstall mechanism. Actually, if you need to remove your feature, then you can set the IsInstalled feature to 0 and change the StubPath so that the command there executes a cleaning task of your choice. At logon if the HKCU subkey of active setup was registered previously then the command in the StubPath will be executed. The versioning allows you to reexecute the ActiveSetup command even if the ActiveSetup has already been executed by the user i.e. if the HKLM ActiveSetup key is already mirrored in the HKCU hive. ActiveSetup is poorly (read not) documented but you may find information on the web for example in this good article.

ActiveSetup set in the HKLM registry for the Wix sample installer

ActiveSetup set in the HKLM registry for the Wix sample installer

Ok now, how we will use this for our Wix Install? In addition to the common HKLM registration, our sample builds a .NET40 executable binary called managedOpenKey.exe. This .exe, when invoked with /install parameter, looks at the HKCU hive and creates the OPEN subkey mentioned in the second paragraph, so that the addin will be opened when starting Excel. When the .exe is invoked with /uninstall it will remove the OPEN value, if present, in the HKCU. This is simple: at install/ repair/upgrade our Wix creates the ActiveSetup HKLM key with a StubPath of the form “manageOpenKey.exe /install “. For uninstall, we set the IsInstalled value to zero and change the StubPath to “manageOpenKey.exe /uninstall”. To do so we use some custom actions written in .NET40 of the Wix installer, they must be invoked without impersonation so that we are allowed to write in HKLM.

We wished to allow the usage of the addin without a reboot. Therefore, at install/repair/upgrade the msi quietly invokes the same manageOpenKey.exe so that, for the current user performing the install we do not need the reboot and wait for the active setup to create the HKCU OPEN value. But there is a trick…. We must force the mirroring of the ActiveSetup key in the HKCU registry. Indeed, take the following example: user 1 installs the addin; the OPEN key is registered because we have invoked the .exe directly in the .msi. Now user1 logs off and user2 logs on, ActiveSetup is triggered for him and the addin is registered and everyone is happy. But now user2 decides to uninstall the addins and does so. The user1 when he logs back and opens Excel will have an error message saying that the addin is not found. What happened? The HKCU key for ActiveSetup was not mirrored in the user1 HKCU therefore ActiveSetup does not consider that the feature was installed for him then it does not trigger the StubPath command for cleaning environment when the IsInstalled has been set to false. Tricky, isn’t it? These manual creation of HKCU key are made by C# custom actions that should be impersonated because you want to write your key in the HKCU of the user who made the install not the LocalAdministrator that runs the custom action not impersonated such as those mentioned in previous paragraph.

There is also a trick regarding the OS bitness. If you do not take care and write the ActiveSetup in the HKLM hive within a 32bit process on a 64bit OS then your active setup will be written under /Wow6432 path. However, when performing the HKCU mirroring mentioned above the .NET API will write in HKCU/Software/Microsoft/Active Setup because it is not supposed to have a Wow6432 key in HKCU. However, there is Wow6432 HKCU key used only by ActiveSetup and you should be cautious. Indeed, the ActiveSetup in /Wow6432 and the one directly under HKLM/Software/Microsoft do not interact. Therefore, in our Wix CustomActions we have to check OS bitness and write the HKLM part in the proper slot with the following code snippet.

To conclude, I would like to thank my colleague Sébastien Cadorel who helped me a lot to write down this install sample. If you encounter any troubles with this install sample feel free to submit a bug or to pull request on the github.

No hdmi sound without OS reboot? Restart the drivers…

Edit: end july 2015 the problem is now fixed for me with windows 10.

This post blog follows a problem that I encountered when I upgraded my laptop to windows 8.1. Precisely, the hdmi sound stops working immediately when I connect the hdmi cable to my television. The hardware works fine because if the OS is reboot with hdmi cable plugged, then the TV as a sound playback device is available. This is terribly annoying, you do not want to reboot each time you want to use your TV as a display device. While searching the internet for a solution, I discovered that I am not the only one with this problem. Therefore, I will try to expose my solution with many details in order to make it reusable by others who would stumble on this post with the same problem.

Assuming that you are in the same, or a close, situation: the hdmi sound works well when Windows is reboot with the cable plugged in. Consequently, when everything is working, if you right click on the sound icon in the tray bar and select “playback devices” you should be able to see the TV in the device playbacks .

The list of available playback devices.

The list of available playback devices.

Before going any further the first thing to do is to update the drivers. This is quite simple, right click on the windows menu at the bottom left of your desktop screen and select the device manager. My advice is to update all drivers for the two entries in the tree view “display adapters” and “sound, video and game controllers”.

The device manager

The device manager

Once all drivers have been updated, check that the problem still occurs (restart the OS once at least) and if it does you may be interested in the following.

Upgrading driver software...

Upgrading driver software

So even after updating the drivers, you still need the reboot to get the sound. Actually, the drivers managing the hdmi need to be restarted and that is what we are going to do manually, without restarting the OS. To perform such action, we are going to need devcon.exe which is a tiny program distributed freely by Microsoft with the Windows Driver Kit. Download it here (you do not need Visual studio).

After the (silent) installation of WDK you may find devcon.exe in the the path “C:\Program Files (x86)\Windows Kits\8.1\Tools\x64”. You will notice that you have two directories in Tools the “/x64” and the “/x86” (i.e 32 bits), use the one corresponding to your system (to know more on 32 bits vs 64 bits check this).

WARNING: devcon.exe error messages are extremely misleading (read completely wrong…). Indeed, if you are using the x86 version instead of the x64 version or if you are trying to restart the drivers (see below) without administrators right you will still have the same error.: “No matching devices found”. Consequently, do not try to interpret the error raised by devcon.exe.

listmedia

“devcon.exe listclass media” executed in powershell using conemu terminal

Now, we are going to restart the two drivers using devcon.exe. First, let us see all drivers dedicated to sound, then type in your command prompt (or better with powershell) run as administrator: <pathToDevCon>/devcon.exe listclass media

The response here is a list of two items, we are going to retain the first one, “Intel (R) display”, which was the audio adapter used for the hdmi (see the first screen shot of this post). The driver Id: HDAUDIOFUNC_01&VEN_8086&DEV_2807&SUBSYS_80860101&REV_10004&36161A1D&0&0001 is going to be needed. If you are using powershell you may use the Out-File command to write it down in a text file…

You can try to restart it directly by typing (in command line run as administrator): <pathToDevCon>/devcon.exe restart “@<DRIVER_ID>”.
Note that the @ symbol is inside the quotes which may surprise you if you are a .NET developer…

Unfortunately, it does not seem to be sufficient and the system also needs to reboot the display adapter driver after that. So we are going to do the same action but for display adapter driver: we type in an admin command line:
<pathToDevCon>/devcon.exe listclass display

devcon.exe listclass display

devcon.exe listclass display

Then now if you restart the driver listed here as “Intel(R) HD Graphics Family” then the HDMI sound device playback should be available if the cable is connected. To do so execute:

<pathToDevCon>/devcon.exe restart “@PCIVEN_8086&DEV_0A16&SUBSYS_130D1043&REV_093&11583659&0&10”

The Id above is mine, obviously you should use yours.

Fine but this is not over yet. Indeed, you do not want to do all this each time you are connecting your hdmi cable with your TV. What we are going to do right now, is to add a powershell script that will automated this process so that you will just have to click on one icon when you will want your TV sound. If you have never run powershell script on your machine, then the execution of ps1 file (powershell script) will not be allowed. Read this to enable powershell script execution (basically you will just have to type in a powershell console, run as admin, Set-ExecutionPolicy Unrestricted).

Here is the script that you can use. I have added two instructions to enable the drivers, it seems to be sometimes useful when you plug/unplug the hdmi cable several times. All you have to do to make it work is to change the path to devcon.exe and the id. You can save it to the place of your choice such as the Desktop with a .ps1 extension (e.g. restart_drivers.ps1)

Remind that to be effective this .ps1 file should be executed with administrator rights. Remind also, that devcon.exe will only tell you that the drivers cannot be found if you are not with admin rights. So I am suggesting to make the script more easily run in admin mode, you can put a powershell.exe shortcut next to the .ps1 script file. By right clicking on the shortcut file go to Properties and fill the target property as follows. Target:”powershell.exe restart_drivers.ps1″. Then, in Advanced… select Run as administator.

A shortcut to run restart_drivers.ps1 directly with admin right

A shortcut to run restart_drivers.ps1 directly with admin right

That is all, no when you will connect your hdmi cable you will only have to click on the shortcut and enjoy the sound from your TV.