Posts by Justin Clift 288 publicly visible posts • joined 1 May 2007
> It can only send that speed if the network infrastructure supports it.
Yeah, of course. 10GbE switches don't seem to be priced yet at general-consumer levels, though they're a lot cheaper these days than they used to be.
Prosumer pricing might be the best description of things currently?
That being said, if noise isn't a factor then 2nd hand 40Gb/s infiniband switches are cheap on Ebay. :)
As a data point, the cpu and ram upgrades arrived yesterday. They're all working well.
The ram is identifying itself as authentic HPE "Smart Memory" (whatever that means), it's passed an extended run in memcheck86+, and is now running real workloads without issue. :)
Looking at those two, why are they better to use than the arc min and max values?
For a single purpose system (NAS or similar), I can sort of see why arc_sys_free would be useful.
On systems with multiple other applications though (and where ZFS performance is irrelevant), being able to set a maximum memory cap for ZFS ARC then never having to think about it again seems better.
Or is that not the full picture? :)
Sure, ping me your contact details via the email address in my GitHub profile and I can let you know. :)
Cool, that's all well reasoned and gives food for thought. Thanks. :)
The setup I'm testing (small hyperconverged cluster using ceph for vm storage), there have definitely been some learning experiences.
It's still way too early in my testing/learning process to be comfortable rolling it out to production, as there are times I've gotten the underlying Ceph storage "stuck" or unresponsive... and ended up having to rebuild the cluster.
But, I'm rapidly getting better at understanding how the underlying pieces operate, then being able to unstick a cluster that's frozen (etc). So I'm hopeful it turn out to be workable. :)
Came across this situation of yours just yesterday, and found an easier solution:
<quote> Accidentally using the ‘restart’ button on a VM that’s guest OS is hung then leads to the hypervisor waiting for it to respond – which it won’t do because it’s hung. And then you can’t use the ‘reset’ option until that’s clear. To clear it on a cluster, you need to SSH into one of the hosts to locate and clear some lock file before you then kill off the process. It’s a silly little niggle in UI design that then sends you right back to the command line for 5 minutes.</quote>
I accidentally did the "restart" thing on a stuck VM yesterday too, which then promptly wedged and blocked subsequent operations.
(At least with Proxmox 8) double clicking the log entry for that initial wedged "restart" operation at the bottom of the proxmox gui opens a progress dialog where you can see it doing nothing. There's a "stop" button in that dialog.
Clicking that stop button (and giving it a few seconds), seems to correctly cancel the wedged restart job, allowing new actions (like a hard power off or whatever) to function.
That being said, I'm still pretty new to Proxmox and have only used Proxmox 8. No idea if that's a new thing or was just unreliable previously etc. :)
> Barely-out-of-homebrew Linux based garbage products *ahem* *proxmox* ...
What issues did you hit with Proxmox?
I've been testing it (often breaking it ;>) in my homelab recently, to see if it'd consider it reliable enough for real world use in a colo place.
Yep, they're indeed a socketed cpu: https://www.servethehome.com/hpe-proliant-microserver-gen10-plus-review-this-is-super/2/
Wish I hadn't missed that before now, as I've seen a few of these models over the years. ;)
> The amount of RAM ZFS requires is just crazy.
Hmmm, "requires" is probably too strong a word.
The default memory settings that OpenZFS uses (especially for ARC max value) really do seem to be a case of "grab everything that's not nailed down".
For stand alone appliances, that's probably ok. But for situations where it's supposed to be on the same boxes as other stuff... you're better off to rein it in.
The two major sysctl settings for that (zfs_arc_min and zfs_arc_max) are easily set on the fly. To limit ARC to 2GB max use this:
# echo 1073741824 > /sys/module/zfs/parameters/zfs_arc_min
# echo 2147483648 > /sys/module/zfs/parameters/zfs_arc_max
To set the values permanently, throw them into a new modprobe options file:
# cat /etc/modprobe.d/zfs.conf
options zfs zfs_arc_min=1073741824
options zfs zfs_arc_max=2147483648
They'll be loaded automatically at boot time from then on.
> ... only the Gen 8 can be affordably upgraded, and even then it would be neither cheap nor easy.
That's kind of funny, as just yesterday I've ordered the upgrades for one of my Gen 8's:
* ECC ram sticks - https://www.ebay.com.au/itm/363949069992 (2 of these)
* CPU upgrade to E3-1265L V2 - https://www.ebay.co.uk/itm/133019746407?var=432313510635
Already upgraded one of my microservers (I have a bunch), and I'll probably do the rest in a few weeks too.
The PCIe x8 slot (only gen 2 though) can even take a 10GbE card, or even dual port 2.5GbE ones if that's more your thing. :)
Took some time today to throw together a new testing setup, and so far the VM migrations are now being reliable.
This time it was 5 nodes for just Proxmox Ceph storage (no hosting of VMs on those), with another 3 nodes for the hosting of VMs (also in Proxmox).
Doing 30 migrations (manual UI button clicking), and they all worked without issue. No random hangs.
That bodes well, so maybe Proxmox is an optional after all. Will need to put some time aside for properly testing it again in the near-ish future. :)
> what made you fail proxmox
Entirely the hanging of VM migrations. It's a central requirement for us, in order to have a better approach to updating host systems without downtime for hosted clients.
The testing was on three separately set up clusters over two months, 1x 3 node cluster, 2 x 5 node clusters. All using Ryzen 5xxx series gear (mostly 5950x).
The hanging of migrations never even leaves any kind of logs for diagnosis. And cancelling a hung migration *kills* (as in stops, like a power off) the VM in question rather than letting it continue on running where it was.
So, using Proxmox is just not a possibility. At least until they have VM migration being reliable. We're just sticking with the existing approach for now, which is pretty much "schedule downtime and update the hosts". Ugh.
One possibility might be to try separate clusters of hosts vs storage. Just in case it was some interaction of having both Ceph and the VMs running on the same boxes (hyperconverged style).
I *might* put some time into testing that, just in case, as having a working solution would be really useful. :)
It's interesting they went with EPYC processors.
In my testing of Proxmox 8.x (with Ryzen 5000 series processors), VM migrations would randomly hang forever about 25% of the time.
That's with a 5 node setup using Ceph storage on the same Proxmox nodes. And it definitely wasn't a case of the network interconnects not being fast enough either.
Saw a mention of an article some weeks ago where they had a similar hanging problem with Proxmox on AMD processors, apparently it was a recurring problem over the years. But there was some kind of patches to fix the problem this time around.
I didn't investigate that solution in depth because I'd already finished up the Proxmox testing (result: fail), and didn't want to waste time setting up+testing the entire cluster again just on the hope maybe things were actually fixed.
Has anyone else come across that hanging thing with AMD cpus on Proxmox?
As a data point, the ASRock B550M Pro4 also supports ECC.
Am using one as my desktop, with a Ryzen 5600X and 4x Kingston KSM26ED8/16HD 16GB ram sticks.
Note that the ram sticks in my system are running fine at 3200MT/s, and dmidecode reports:
Error Correction Type: Multi-bit ECC
and
Total Width: 72 bits
Data Width: 64 bits
Didn't see a link to the video itself in the article, but it seems to be online now:
> platform CSV exporter
Errr.... which platform? Almost every CSV exporter is custom build.
Also note, "the CSV RFC" is not as clear cut as you're making it out to be.
It was only an attempt to draw a sand in the line as there were so many incompatible implementations already floating around.
That RFC does cover most of the common stuff, but doesn't cover a few key areas (null values? binary encoding? unicode bom? field names?).
There have been some follow up concept docs started since, but nothing has become a new & improved RFC yet either.
> I've never been disappointed by CentOS ...
No KDE in RHEL7, so unless there's a solid 3rd party repo created for it, CentOS7 is the last version of CentOS I'll be running as a desktop. :(
Interestingly, OpenSUSE Leap looks like it might do the job as a replacement adequately instead.
> In this case, unless you DID check beforehand, the error was on YOUR side for taking the word of a front desk "sales girl" about the bonus before putting in the effort.
Oh, they had leaflets to the same effect for her to put up. Wasn't just her imagination. ;)
> I'm not trying to advocate a monopoly, but the market data does not look good for Mellanox in spite it being the superior product.
Mellanox had a very large headway - nearly a decade - to make Infiniband popular.
Unfortunately, they did their level best to kill all possible grass roots uptake by ensuring SysAdmin's, Hobbiests, etc couldn't use the gear unless it came directly from them.
Mellanox (on purpose) drop driver support for any cards that are not current models for sale, and remove their older drivers.
So, the large supply of super cheap Mellanox cards cards on Ebay (for years)... were useless for the majority of people.
It's a good thing FreeBSD and Linux included working drivers in the OS distribution, as if it was up to Mellanox they'd have stopped that too.
Ask a question on their "Community" forums about a non-current model, and you're told it's an old card, ending the "conversation".
End result... most SysAdmin's & System Architects haven't ever played with Infiniband, so when the time comes to include or recommend network gear for new projects, guess what's never included as it's "exotic"? The only place it then gets a look in, is for things where "exotic" is ok.
A decade of consistently doing this, and it's literally no wonder Infiniband never "caught on" despite being the better tech. Friggin Morons for their "Sales and Marketing" leadership. "Short term thinking" played out for a decade.
> And then there are all the more aggressive options. Yes, there is miniscule and mitigatable probability of property damage where the bullet might land - what is that damage likely to be vs. economic damage caused?
Not seeing a need to use bullets?
If the problem drone can be seen clearly enough to shoot it, then why don't the authorities just grab some drones themselves (from any kids store!) and ram those into the thing?
Sure, kids store drones are unlikely to be strong. But several (say 10) of the things should do the trick.
If they can't find someone with the required piloting skills... ask some kids. Pretty sure they'll be up for it. :)
> JFrog anticipates rapid growth of the library as the community takes advantage of the freebie service, reckoning the repository will soon be home to thousands of modules, including packages for tech such as Kubernetes.
So, their "modules" are going to be precompiled bits of Go code? Which developers should trust, instead of just compiling the same code themselves. Which Go would compile very quickly anyway.
Frankly, really not seeing the point. It sounds like JFrog are trying to insert themselves into the Go ecosystem, but haven't really figured out where they could do so usefully.
> Go lacks a central, public repository for modules.
No, it really doesn't.
The default central, public repository for modules is GitHub. Developers are free to place their code in other places too, but GitHub is the generally accepted source. Also happens to be version controlled.
> Without an immutable source for code, developers have to repackage their modules
Huh? Is someone getting mixed up with Python?
> ... and, rightly, eye modules pulled off the 'net with suspicion since the code could easily change without warning.
How is having your code in JFrog going to be more trustworthy than (the same code) from GitHub?
> ... but some residual value in a Mac, and resale will mean disk wiping, and refurbishment, ...
It'll be interesting to see if the "glued in" approach to disks means the whole Mac mac will need shredding for data security.
If the storage really can't be practically replaced, anyone using Mac's in (at least) reasonably secure environments isn't going to find much resale value in them.
Epson has some "bottles of ink" friendly printers:
https://epson.com/ecotank-super-tank-printers
Some of the models there print A3 and bigger.
Haven't tried any of them personally though.
> ... recently noted that HiFive RISC-V chips have proprietary pieces.
SiFive (the maker of the HiFive) apparently got the message, and is putting in the extra effort in to open up the rest.
https://www.phoronix.com/scan.php?page=news_item&px=SiFive-Open-Boot-Code-Coming
Not sure if that's actually happened yet or not.
Hopefully it has, or does soon. :)
While some of the frothing-at-the-mouth OSS advocates ;) will blindly dismiss this as bad, it's an attempt to solve a real problem.
Sure, it may go poorly. But also it may work, or provide useful information to Redis to determine a real working solution.
Let's give them some time to see how it goes in the real world. If they turn out to be idiots about it, we can grill them later. ;)
> As a trivial example, I have developed Java applications on x86 then run them on ARM (on a Raspberry Pi). I have also written Qt C/C++ applications on x86 and recompiled them for ARM.
Go (the language) will have ARMv8 support in the next release (1.11). Been testing it already in production deployment with the recent betas, and it works well.
> But I would recommend buying the kit to learn about how it works...
Yeah. I bought a FlashForge Creator Pro. 2015 model from memory. When it started acting up, I had no clue at all how to fix it.
Went and bought a Shapeoko 3 (CNC) instead, and made sure it was a kit so I learned how the bits all went together. As you mention, now I can build and upgrade stuff reasonably effectively. :)
> "We don't need a £2000 key ring. And you don't play chess!"
That sounds familiar. :)
I tried out the 3D printing thing, but wasn't impressed by the end result. Have since gotten into doing CNC stuff instead. Today's task was to make full height brackets for some cheapo 10GbE Mellanox cards picked up on Ebay.
Turned out pretty well I reckon. Much more rugged than the 3D printed bracket version. :)
Photo's here, in case it's interesting.
Printrbot closed their doors a few days ago too. :(
> So you need something with the speed of a computer and the subjectivity of a human ...
Needs wisdom too, and someone to keep an eye on it's developing nature. Something like you suggest that - for example - starts leaning towards various forms of intolerance / racism / badness-in-general could go pear shaped very quickly.
> the SF that was embedding extra stuff - can't remember what, ...
They were bundling malware with the Windows downloads of popular Open Source projects, and giving the developers a cut of the profits.
FileZilla comes to mind, their forums have many archived/closed threads from people negatively affected (eg spyware installed on the PCs) but the Filezilla admin just stuck their fingers in their ears and went "la la la".
Disgusting behaviour all around.
"browser.pocket.enable" doesn't seem to exist in Firefox 52.x (ESR), on CentOS 7.x.
"extensions.pocket.enabled" does though, and seems like the right one.
> Dammit, I still have i7-920's in use. Fortunately, not on the public interwebs though. And now I'd better make sure they never are.
Fuck. Just checked, and my main gaming rig is on the list too. It's an Intel Core2 Extreme X9650. It does absolutely fine for the stuff I use, and there's no damn way it's "too slow", etc.
Intel, you'd better think again. You screw this up, it's on you to fix it.
> BUT they have already chosen a processor board they would like to use (an A20-OLinuXino-LIME2). ...
Sure. They've definitely jumped onto the bits they feel comfortable with, and obviously have not much clue with the rest.
However, it does sound like they'd be open to constructive pointers telling them they pieces they need to clue up on. You obviously have depth in areas they lack but need.
As an idea, maybe point out the electrical bits they need in order to not completely burn their own houses down ;), and see if they manage to get something useful for people happening after all.
Their skills are stronger in software, so they might turn out to have a decent software side to things anyway. :)
> And, as an avid PC gamer, I've yet to see a single game demanding over 16GB+ RAM.
Highest RAM requirements I've seen to date is for Star Citizen:
Windows 7 (64bit) with Service Pack 1, Windows 8 (64bit), Windows 10 - Anniversary Update (64bit)
* DirectX 11 Graphics Card with 2GB RAM (4GB strongly recommended)
* Quad Core CPU
* 16GB+ RAM
* SSD strongly recommended
Haven't tried it out, so no idea personally what the performance is like with various hardware configs (eg <16GB, 16GB, 32GB, etc).
> > Nothing you can't setup on your own with free tools, if you don't want to pay, and get better ones with far more control.
> Some links would be helpful.
Gitea is a good start. Decent UI, and very lightweight on resources. eg can be run effectively on Raspberry Pi style hardware, though for real business use you'd want it on something proper. :)
GitLab has more features than Gitea, though it's user interface fairly sucks and it's a resource pig (written in Ruby). It can also grow into a PITA to admin over time if your needs aren't basic.
Pick whichever takes your fancy, or do some searching online for others. The above two aren't the only ones. :)
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
