Posts tagged with "dell-wyse-3040"

I found out awhile ago that eMMC storage is a different thing entirely when it comes to health monitoring. This is especially true when you’re booting from it like on the Dell Wyse 3040s of which I have several in my homelab. The goal is to get some status information on the eMMC storage health, but the usual SMART utilities don’t work on eMMC.

root@pve1:~# smartctl -H /dev/mmcblk0 -d auto
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.8.12-1-pve] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

/dev/mmcblk0: Unable to detect device type
Please specify device type with the -d option.

Turns out eMMC has its own health monitoring system that’s actually pretty useful once you know how to access it. I figured I would share my experience since it took some time to figure out. Maybe this will help someone else.

Performance tuning Ceph in a homelab environment presents unique challenges, especially when running on USB storage and constrained hardware. After dealing with performance issues during cluster rebalancing and OSD expansion, I’ve learned valuable lessons about mClock configuration, IOPS optimization, and the realities of USB 3.0 storage performance.

When running Ceph in a homelab environment, especially on resource-constrained hardware like my Dell Wyse 3040 cluster, managing storage capacity becomes critical. Understanding Ceph’s Nearfull warnings and how to respond to them can prevent your cluster from going read-only unexpectedly.

I’ve been running Proxmox in my homelab since version 7.4, and the journey to Proxmox 8.2.2 was to say the least… educational. Let me share some hard-won lessons that might save you some headaches. These even apply to the Proxmox 9 upgrades as well which I have not scheduled in my cluster yet. I’m pretty sure I’ll have updates when I get to that upgrade to share.

My super lean Proxmox 8.3 testbed cluster running Ceph occasionally just decides to lockup a node based on it being incredibly limited on RAM and CPU. As much as I hate rebooting Linux/UNIX systems, this is a case where a nightly reboot of the nodes might help with reliability.

My earlier post for ProxMox 8.2.2 Cluster on Dell Wyse 3040s mentioned the tight constraints of the cluster both with RAM and DISK space. There are some extra steps involved in keeping a very lean Proxmox 8 cluster running on these extremely resource limited boxes. I am running Proxmox 8.2 and Ceph Reef on them which leaves them slightly under resourced as a default. So when the Ceph would not start up the Ceph Monitors after my upgrade from Proxmox 8.2.2 to 8.2.4, I had to dig a bit to find the problem.

Ceph Monitor will not start up if there is not at least 5% free disk space on the root partition. My root volumes were sitting right at 95% used. So our story begins…

I have been using the Dell Wyse 3040 as awesome little systems for my Tailscale nodes in my multiple joint homelab networks. These systems are super low power consuming and physically small enough to just plug and go. Truly, deploying a WireGuard®-based VPN solution could not be any easier. I have four of these units connecting my homelab networks across three geographically diverse locations.

I want a place to test and try out new features and capabilities in Proxmox 8.2.2 SDN (Software Defined Networking). I would also like to be able to test some Ceph Cluster configuration changes that are risky as well. I do not want to do it on my semi-production Proxmox 8.2.2 Ceph enabled Cluster that I have mentioned in earlier posts. With 55TiB of raw storage and 29TiB of it loaded up with content, that would be painful to rebuild or reload if I made a mistake during my testing of SDN or Ceph capabilities.

The Dell Wyse 3040 is a nifty little machine that is extremely small and a low power consumer. They are however not without issues. This is my foray into trying to get a couple of them working on my homelab. I bought a couple in early March 2024 based on watching Apalrd’s videos Dell Wyse 3040 Thin Client Teardown and Installing Proxmox VE 7 on Debian Bullseye with the various Dell Wyse thin clients.

I have collected nine (9) mostly functional Dell Wyse 3040 thin clients for use in my experimentation with Proxmox Clusters and SDN and Site-2-Site VPN configurations with Tailscale. Yes, I know I have a problem. :)

On the upside, they are very small low power consuming Debian 12 servers that have a 1Gbps NIC and run headless nicely once you fix the BIOS settings and Debian configuration correctly. What is not nice is their CMOS batteries are all mostly dying on me and their connector is a odd type that is not supported by many vendors and are between $8-$12 USD to replace. For example the Rome Tech CR2032 CMOS BIOS Battery for Dell Wyse 3040 is about $9.89 USD as of posting this. This bothers me intensely as the actual CR2032 can be picked up for well under a dollar ($1 USD) each at LiCB CR2032 3V Lithium Battery(10-pack) for a pack of 10 for $6 USD. Also, I’m picking these units up with power adapter for between $20 and $45 on eBay and the $10 bite jacks my price per unit up a good bit. So what to do?