* Posts by Justin Clift

288 publicly visible posts • joined 1 May 2007

Page:

Kaby Lake-G chip back from the grave, now on modest firewall-router-NAS mobo

Justin Clift

Re: Router motherboard, not a NAS motherboard

> It can only send that speed if the network infrastructure supports it.

Yeah, of course. 10GbE switches don't seem to be priced yet at general-consumer levels, though they're a lot cheaper these days than they used to be.

Prosumer pricing might be the best description of things currently?

That being said, if noise isn't a factor then 2nd hand 40Gb/s infiniband switches are cheap on Ebay. :)

TrueNAS CORE 13 is the end of the FreeBSD version

Justin Clift

Re: Doesn't TrueNAS Scale work just as well on the HP Microservers?

As a data point, the cpu and ram upgrades arrived yesterday. They're all working well.

The ram is identifying itself as authentic HPE "Smart Memory" (whatever that means), it's passed an extended run in memcheck86+, and is now running real workloads without issue. :)

Justin Clift

Re: the caching services zfs requires

Looking at those two, why are they better to use than the arc min and max values?

For a single purpose system (NAS or similar), I can sort of see why arc_sys_free would be useful.

On systems with multiple other applications though (and where ZFS performance is irrelevant), being able to set a maximum memory cap for ZFS ARC then never having to think about it again seems better.

Or is that not the full picture? :)

Justin Clift

Re: the caching services zfs requires

Sure, ping me your contact details via the email address in my GitHub profile and I can let you know. :)

Justin Clift

Re: Oracle doesn't have a ZFS implementation that could run on Linux

> Oracle could run the same ZFS on Linux that everybody else is running. Why don’t they?

Politics.

Justin Clift

Re: the caching services zfs requires

Cool, that's all well reasoned and gives food for thought. Thanks. :)

The setup I'm testing (small hyperconverged cluster using ceph for vm storage), there have definitely been some learning experiences.

It's still way too early in my testing/learning process to be comfortable rolling it out to production, as there are times I've gotten the underlying Ceph storage "stuck" or unresponsive... and ended up having to rebuild the cluster.

But, I'm rapidly getting better at understanding how the underlying pieces operate, then being able to unstick a cluster that's frozen (etc). So I'm hopeful it turn out to be workable. :)

Came across this situation of yours just yesterday, and found an easier solution:

<quote> Accidentally using the ‘restart’ button on a VM that’s guest OS is hung then leads to the hypervisor waiting for it to respond – which it won’t do because it’s hung. And then you can’t use the ‘reset’ option until that’s clear. To clear it on a cluster, you need to SSH into one of the hosts to locate and clear some lock file before you then kill off the process. It’s a silly little niggle in UI design that then sends you right back to the command line for 5 minutes.</quote>

I accidentally did the "restart" thing on a stuck VM yesterday too, which then promptly wedged and blocked subsequent operations.

(At least with Proxmox 8) double clicking the log entry for that initial wedged "restart" operation at the bottom of the proxmox gui opens a progress dialog where you can see it doing nothing. There's a "stop" button in that dialog.

Clicking that stop button (and giving it a few seconds), seems to correctly cancel the wedged restart job, allowing new actions (like a hard power off or whatever) to function.

That being said, I'm still pretty new to Proxmox and have only used Proxmox 8. No idea if that's a new thing or was just unreliable previously etc. :)

Justin Clift

Re: the caching services zfs requires

> Barely-out-of-homebrew Linux based garbage products *ahem* *proxmox* ...

What issues did you hit with Proxmox?

I've been testing it (often breaking it ;>) in my homelab recently, to see if it'd consider it reliable enough for real world use in a colo place.

Justin Clift

Re: Doesn't TrueNAS Scale work just as well on the HP Microservers?

Yep, they're indeed a socketed cpu: https://www.servethehome.com/hpe-proliant-microserver-gen10-plus-review-this-is-super/2/

Wish I hadn't missed that before now, as I've seen a few of these models over the years. ;)

Justin Clift

Re: the caching services zfs requires

> The amount of RAM ZFS requires is just crazy.

Hmmm, "requires" is probably too strong a word.

The default memory settings that OpenZFS uses (especially for ARC max value) really do seem to be a case of "grab everything that's not nailed down".

For stand alone appliances, that's probably ok. But for situations where it's supposed to be on the same boxes as other stuff... you're better off to rein it in.

The two major sysctl settings for that (zfs_arc_min and zfs_arc_max) are easily set on the fly. To limit ARC to 2GB max use this:

# echo 1073741824 > /sys/module/zfs/parameters/zfs_arc_min

# echo 2147483648 > /sys/module/zfs/parameters/zfs_arc_max

To set the values permanently, throw them into a new modprobe options file:

# cat /etc/modprobe.d/zfs.conf

options zfs zfs_arc_min=1073741824

options zfs zfs_arc_max=2147483648

They'll be loaded automatically at boot time from then on.

Justin Clift

Re: Doesn't TrueNAS Scale work just as well on the HP Microservers?

> ... only the Gen 8 can be affordably upgraded, and even then it would be neither cheap nor easy.

That's kind of funny, as just yesterday I've ordered the upgrades for one of my Gen 8's:

* ECC ram sticks - https://www.ebay.com.au/itm/363949069992 (2 of these)

* CPU upgrade to E3-1265L V2 - https://www.ebay.co.uk/itm/133019746407?var=432313510635

Already upgraded one of my microservers (I have a bunch), and I'll probably do the rest in a few weeks too.

The PCIe x8 slot (only gen 2 though) can even take a 10GbE card, or even dual port 2.5GbE ones if that's more your thing. :)

DARPA tasks Northrop Grumman with drafting lunar train blueprints

Justin Clift

> The lunar rail project is part of DARPA's 10-Year Lunar Architecture (LunA-10) project that aims to develop [...] interoperable standards ...

It'll be using metric like the rest of the space community then yeah? :)

Meet the Proxinator: A hyperbox that puts SATA at the heart of VMware migrations

Justin Clift

Re: Proxmox 8.x VM migrations hang with Ryzen gear

Took some time today to throw together a new testing setup, and so far the VM migrations are now being reliable.

This time it was 5 nodes for just Proxmox Ceph storage (no hosting of VMs on those), with another 3 nodes for the hosting of VMs (also in Proxmox).

Doing 30 migrations (manual UI button clicking), and they all worked without issue. No random hangs.

That bodes well, so maybe Proxmox is an optional after all. Will need to put some time aside for properly testing it again in the near-ish future. :)

Justin Clift

Re: Proxmox 8.x VM migrations hang with Ryzen gear

> what made you fail proxmox

Entirely the hanging of VM migrations. It's a central requirement for us, in order to have a better approach to updating host systems without downtime for hosted clients.

The testing was on three separately set up clusters over two months, 1x 3 node cluster, 2 x 5 node clusters. All using Ryzen 5xxx series gear (mostly 5950x).

The hanging of migrations never even leaves any kind of logs for diagnosis. And cancelling a hung migration *kills* (as in stops, like a power off) the VM in question rather than letting it continue on running where it was.

So, using Proxmox is just not a possibility. At least until they have VM migration being reliable. We're just sticking with the existing approach for now, which is pretty much "schedule downtime and update the hosts". Ugh.

One possibility might be to try separate clusters of hosts vs storage. Just in case it was some interaction of having both Ceph and the VMs running on the same boxes (hyperconverged style).

I *might* put some time into testing that, just in case, as having a working solution would be really useful. :)

Justin Clift

Proxmox 8.x VM migrations hang with Ryzen gear

It's interesting they went with EPYC processors.

In my testing of Proxmox 8.x (with Ryzen 5000 series processors), VM migrations would randomly hang forever about 25% of the time.

That's with a 5 node setup using Ceph storage on the same Proxmox nodes. And it definitely wasn't a case of the network interconnects not being fast enough either.

Saw a mention of an article some weeks ago where they had a similar hanging problem with Proxmox on AMD processors, apparently it was a recurring problem over the years. But there was some kind of patches to fix the problem this time around.

I didn't investigate that solution in depth because I'd already finished up the Proxmox testing (result: fail), and didn't want to waste time setting up+testing the entire cluster again just on the hope maybe things were actually fixed.

Has anyone else come across that hanging thing with AMD cpus on Proxmox?

Linus Torvalds's faulty memory (RAM, not wetware) slows kernel development

Justin Clift

Another ECC board

As a data point, the ASRock B550M Pro4 also supports ECC.

Am using one as my desktop, with a Ryzen 5600X and 4x Kingston KSM26ED8/16HD 16GB ram sticks.

Note that the ram sticks in my system are running fine at 3200MT/s, and dmidecode reports:

Error Correction Type: Multi-bit ECC

and

Total Width: 72 bits

Data Width: 64 bits

Neuroscientist used brainhack. It's super effective! Oh, and disturbingly easy

Justin Clift

BSides LV Video is now online

Didn't see a link to the video itself in the article, but it seems to be online now:

https://youtu.be/nEgb-BpYbRw?t=19257

The Eldritch Horror of Date Formatting is visited upon Tesco

Justin Clift

Re: Dates? Don't talk to me about dates...

> platform CSV exporter

Errr.... which platform? Almost every CSV exporter is custom build.

Also note, "the CSV RFC" is not as clear cut as you're making it out to be.

It was only an attempt to draw a sand in the line as there were so many incompatible implementations already floating around.

That RFC does cover most of the common stuff, but doesn't cover a few key areas (null values? binary encoding? unicode bom? field names?).

There have been some follow up concept docs started since, but nothing has become a new & improved RFC yet either.

How much open source is too much when it's in Microsoft's clutches? Eclipse Foundation boss sounds note of alarm

Justin Clift

Re: "Where all the "cool ones" were" ? Wrong.

> Sourceforge was not the only online code repository available.

It was probably the widest known, until GitHub (and similar git-using systems) came about with a focus in usability. SourceForge, Savannah, etc, all have terrible usability. :(

Can I get a RHEL yeah? Version 8 arrives at last as IBM given go-ahead to wolf down Red Hat

Justin Clift

Re: Centos 8

> I've never been disappointed by CentOS ...

No KDE in RHEL7, so unless there's a solid 3rd party repo created for it, CentOS7 is the last version of CentOS I'll be running as a desktop. :(

Interestingly, OpenSUSE Leap looks like it might do the job as a replacement adequately instead.

IT sales star wins $660k lawsuit against Oracle in Qatar – but can't collect because the Oracle he sued suddenly vanished

Justin Clift

Re: I just think this is the way sales companies work

> In this case, unless you DID check beforehand, the error was on YOUR side for taking the word of a front desk "sales girl" about the bonus before putting in the effort.

Oh, they had leaflets to the same effect for her to put up. Wasn't just her imagination. ;)

The Six Billion Dollar LAN: Intel hopes to gobble network kit biz Mellanox 'for $6bn'

Justin Clift

Re: Monopoly?

> I'm not trying to advocate a monopoly, but the market data does not look good for Mellanox in spite it being the superior product.

Mellanox had a very large headway - nearly a decade - to make Infiniband popular.

Unfortunately, they did their level best to kill all possible grass roots uptake by ensuring SysAdmin's, Hobbiests, etc couldn't use the gear unless it came directly from them.

Mellanox (on purpose) drop driver support for any cards that are not current models for sale, and remove their older drivers.

So, the large supply of super cheap Mellanox cards cards on Ebay (for years)... were useless for the majority of people.

It's a good thing FreeBSD and Linux included working drivers in the OS distribution, as if it was up to Mellanox they'd have stopped that too.

Ask a question on their "Community" forums about a non-current model, and you're told it's an old card, ending the "conversation".

End result... most SysAdmin's & System Architects haven't ever played with Infiniband, so when the time comes to include or recommend network gear for new projects, guess what's never included as it's "exotic"? The only place it then gets a look in, is for things where "exotic" is ok.

A decade of consistently doing this, and it's literally no wonder Infiniband never "caught on" despite being the better tech. Friggin Morons for their "Sales and Marketing" leadership. "Short term thinking" played out for a decade.

London's Gatwick airport suspends all flights after 'multiple' reports of drones

Justin Clift

Re: Multiple drones/operators/battery packs?

> And then there are all the more aggressive options. Yes, there is miniscule and mitigatable probability of property damage where the bullet might land - what is that damage likely to be vs. economic damage caused?

Not seeing a need to use bullets?

If the problem drone can be seen clearly enough to shoot it, then why don't the authorities just grab some drones themselves (from any kids store!) and ram those into the thing?

Sure, kids store drones are unlikely to be strong. But several (say 10) of the things should do the trick.

If they can't find someone with the required piloting skills... ask some kids. Pretty sure they'll be up for it. :)

You better watch out, you better not cry. Better not pout, I'm telling you why: SQLite vuln fixes are coming to town

Justin Clift

Bug is in the FTS3 module

As a data point, the bug in question is in the FTS3 module.

If you're using a version of SQLite which doesn't have the FTS3 module, then this bug won't be affecting you.

JFrog to open freebie central repository for Go fans in the new year

Justin Clift

(continued)

> JFrog anticipates rapid growth of the library as the community takes advantage of the freebie service, reckoning the repository will soon be home to thousands of modules, including packages for tech such as Kubernetes.

So, their "modules" are going to be precompiled bits of Go code? Which developers should trust, instead of just compiling the same code themselves. Which Go would compile very quickly anyway.

Frankly, really not seeing the point. It sounds like JFrog are trying to insert themselves into the Go ecosystem, but haven't really figured out where they could do so usefully.

Justin Clift

So... trying to replace GitHub?

> Go lacks a central, public repository for modules.

No, it really doesn't.

The default central, public repository for modules is GitHub. Developers are free to place their code in other places too, but GitHub is the generally accepted source. Also happens to be version controlled.

> Without an immutable source for code, developers have to repackage their modules

Huh? Is someone getting mixed up with Python?

> ... and, rightly, eye modules pulled off the 'net with suspicion since the code could easily change without warning.

How is having your code in JFrog going to be more trustworthy than (the same code) from GitHub?

Sysadmin running a Mac fleet? IBM has just thrown you a lifeline

Justin Clift

Re: So...

> ... but some residual value in a Mac, and resale will mean disk wiping, and refurbishment, ...

It'll be interesting to see if the "glued in" approach to disks means the whole Mac mac will need shredding for data security.

If the storage really can't be practically replaced, anyone using Mac's in (at least) reasonably secure environments isn't going to find much resale value in them.

HP Ink should cough up $1.5m for bricking printers using unofficial cartridges – lawsuit

Justin Clift

Re: When do we get compensation in the UK?

Epson has some "bottles of ink" friendly printers:

https://epson.com/ecotank-super-tank-printers

Some of the models there print A3 and bigger.

Haven't tried any of them personally though.

Boffins are building an open-source secure enclave on RISC-V

Justin Clift

RISC-V HiFive Open-ness

> ... recently noted that HiFive RISC-V chips have proprietary pieces.

SiFive (the maker of the HiFive) apparently got the message, and is putting in the extra effort in to open up the rest.

https://www.phoronix.com/scan.php?page=news_item&px=SiFive-Open-Boot-Code-Coming

Not sure if that's actually happened yet or not.

Hopefully it has, or does soon. :)

Redis has a license to kill: Open-source database maker takes some code proprietary

Justin Clift

Wait and see

While some of the frothing-at-the-mouth OSS advocates ;) will blindly dismiss this as bad, it's an attempt to solve a real problem.

Sure, it may go poorly. But also it may work, or provide useful information to Redis to determine a real working solution.

Let's give them some time to see how it goes in the real world. If they turn out to be idiots about it, we can grill them later. ;)

Microsoft Visual Studio C++ Runtime installers were built to fail

Justin Clift

Re: Probably finding it difficult to recreate the installers

> I put my own installer on github a while back.

url?

Heads up: Fujitsu tips its hand to reveal exascale Arm supercomputer processor – the A64FX

Justin Clift

Re: Why no ARM servers?

> As a trivial example, I have developed Java applications on x86 then run them on ARM (on a Raspberry Pi). I have also written Qt C/C++ applications on x86 and recompiled them for ARM.

Go (the language) will have ARMv8 support in the next release (1.11). Been testing it already in production deployment with the recent betas, and it works well.

FBI boss: We went to the Moon, so why can't we have crypto backdoors? – and more this week

Justin Clift

Re: Man on the sun

> 'if we can put a man on the moon, surely we can put a man on the sun,'

Whichever volunteer gets chosen for that will be a star... err... for the rest of their life. :)

If only 3D desktop printers could 3D print sales! Units crash in Q1

Justin Clift

Re: If only 3D Printers could print 3D Printers

> Would that count as reproduction?

If it was automated, then probably yes.

Extra bonus points if several printers "get together" to combine their features into the subsequently er... spawned ones.

Justin Clift

> But I would recommend buying the kit to learn about how it works...

Yeah. I bought a FlashForge Creator Pro. 2015 model from memory. When it started acting up, I had no clue at all how to fix it.

Went and bought a Shapeoko 3 (CNC) instead, and made sure it was a kit so I learned how the bits all went together. As you mention, now I can build and upgrade stuff reasonably effectively. :)

Justin Clift

> "We don't need a £2000 key ring. And you don't play chess!"

That sounds familiar. :)

I tried out the 3D printing thing, but wasn't impressed by the end result. Have since gotten into doing CNC stuff instead. Today's task was to make full height brackets for some cheapo 10GbE Mellanox cards picked up on Ebay.

Turned out pretty well I reckon. Much more rugged than the 3D printed bracket version. :)

Photo's here, in case it's interesting.

Justin Clift

Printrbot now closed too

Printrbot closed their doors a few days ago too. :(

Science! Luminescent nanocrystals could lead to multi-PB optical discs

Justin Clift

Re: NAND is so nice

> I doubt writeable DVDs are stable.

M-DISC is widely available these days, and caters to those needing archival quality storage.

UN's freedom of expression top dog slams European copyright plans

Justin Clift

> So you need something with the speed of a computer and the subjectivity of a human ...

Needs wisdom too, and someone to keep an eye on it's developing nature. Something like you suggest that - for example - starts leaning towards various forms of intolerance / racism / badness-in-general could go pear shaped very quickly.

Microsoft commits: We're buying GitHub for $7.5 beeeeeeellion

Justin Clift

Re: RIP Github

> the SF that was embedding extra stuff - can't remember what, ...

They were bundling malware with the Windows downloads of popular Open Source projects, and giving the developers a cut of the profits.

FileZilla comes to mind, their forums have many archived/closed threads from people negatively affected (eg spyware installed on the PCs) but the Filezilla admin just stuck their fingers in their ears and went "la la la".

Disgusting behaviour all around.

TSB's middleware nightmare: Execs grilled on Total Sh*tshow at Bank

Justin Clift

DR?

Wonder how their new DR site is going.

You know, the one they're legally obliged to have at all times in case of disaster.

If they haven't scoped gotten their primary site to a decent state, there's practically zero chance of them having DR coverage at the moment.

Firefox to feature sponsored content as of next week

Justin Clift

Re: Which reminds me...

"browser.pocket.enable" doesn't seem to exist in Firefox 52.x (ESR), on CentOS 7.x.

"extensions.pocket.enabled" does though, and seems like the right one.

Intel admits a load of its CPUs have Spectre v2 flaw that can't be fixed

Justin Clift

Re: You b..... you just killed Bloomfield!

> Dammit, I still have i7-920's in use. Fortunately, not on the public interwebs though. And now I'd better make sure they never are.

Fuck. Just checked, and my main gaming rig is on the list too. It's an Intel Core2 Extreme X9650. It does absolutely fine for the stuff I use, and there's no damn way it's "too slow", etc.

Intel, you'd better think again. You screw this up, it's on you to fix it.

Rant launches Eric Raymond's next project: Open-source the UPS

Justin Clift

Re: Lack of clue

> BUT they have already chosen a processor board they would like to use (an A20-OLinuXino-LIME2). ...

Sure. They've definitely jumped onto the bits they feel comfortable with, and obviously have not much clue with the rest.

However, it does sound like they'd be open to constructive pointers telling them they pieces they need to clue up on. You obviously have depth in areas they lack but need.

As an idea, maybe point out the electrical bits they need in order to not completely burn their own houses down ;), and see if they manage to get something useful for people happening after all.

Their skills are stronger in software, so they might turn out to have a decent software side to things anyway. :)

Teensy plastic shields are the big new thing in 2018's laptop crop

Justin Clift

Re: @Lee D

> And, as an avid PC gamer, I've yet to see a single game demanding over 16GB+ RAM.

Highest RAM requirements I've seen to date is for Star Citizen:

Windows 7 (64bit) with Service Pack 1, Windows 8 (64bit), Windows 10 - Anniversary Update (64bit)

* DirectX 11 Graphics Card with 2GB RAM (4GB strongly recommended)

* Quad Core CPU

* 16GB+ RAM

* SSD strongly recommended

Haven't tried it out, so no idea personally what the performance is like with various hardware configs (eg <16GB, 16GB, 32GB, etc).

NASA finds satellite, realises it has lost the software and kit that talk to it

Justin Clift

> We were then told that the bean counters had terminated the storage agreement on the grounds that it was an unnecessary expense.

What happened when you submitted the expense form for replacing all of the lost items?

GitHub shrugs off drone maker DJI's crypto key DMCA takedown effort

Justin Clift

Re: "github provides many workflow features"

> > Nothing you can't setup on your own with free tools, if you don't want to pay, and get better ones with far more control.

> Some links would be helpful.

Gitea is a good start. Decent UI, and very lightweight on resources. eg can be run effectively on Raspberry Pi style hardware, though for real business use you'd want it on something proper. :)

GitLab has more features than Gitea, though it's user interface fairly sucks and it's a resource pig (written in Ruby). It can also grow into a PITA to admin over time if your needs aren't basic.

Pick whichever takes your fancy, or do some searching online for others. The above two aren't the only ones. :)

Page: