* Posts by sed gawk

732 posts • joined 1 May 2008

Page:

HashiCorp runs low on staff, calls a halt to Terraform pull requests

sed gawk Silver badge

Re: Ansible? Really?

I don't claim yaml is my favourite, but ansible is quite useful for building AMI / KVM images. Modules in python are fairly quick to put together..

I use packer+ansible+terraform+k8s which is truly a cursed stack.

k8s has decreed the world is yaml except when it's json, it's not quite a DSL, kustomise stretches that..

The big tech companies are jumping on open source, which is great, so long as they comply with the license. A richer pool of contributors is all to the good.

I pay AWS because of IAM, it's the killer feature. I don't care who does the dev work. That's a flaw in Elastic/Dockers model.

Hashicorp/Terraform have a problem in that they offer nothing but the bare bones of the configuration management problem. All the real value is in the providers which other people maintain and provide opensource.

Ordering is not that well handled by terraform once the codebase grows to a reasonable size.

So TF is useful, and does a job, but it's eminently replaceable.

AWS providing integration with IAM makes a service more attractive than a third party offering exactly the same service *to customers of AWS*.

AWS seem to do quite well with supporting open source access to their stack, yes they keep their orchestration tooling in house, but otherwise I think they do disclose a fair amount of useful knowledge about operating systems at vast scale, and managing the security consolidations inherent in that practice.

sed gawk Silver badge

Was "I'll put my bias upfront" too subtle for you?

They struggle to find people to work in a language which is a productivity thief, I think that's related to the language and the domain. You might disagree.

I do think the business case for using golang is extremely shaky. I'd want to know why you can't use Python/Ruby/Java ?

If performance is your answer, fantastic, show me your use case where performance is dominated by your workload, and golang is acceptable.

I notice the abuse aimed at the users of the language originated in your post, I simply find it a poor quality effort with little abilty to produce correct concise code.

It's ideal if you have 10,000 worker ants, but for small teams - 1 - 5 people, use a scripting language for the job.

sed gawk Silver badge

Manglement and Crap pay I can buy, but Terraform is further ahead in the hype stakes than ansible. Being boring (read reliable) and using python and templating, Ansible is bit too useful and stable to be that popular with the medium crowd.

sed gawk Silver badge

I'd take Java or Python, Ruby, maybe not perl, C++ probably not C, Object Pascal, it's slow enough that JS is workable, but not my preference.

The codebase is boilerplatey and the implementation language is repetitive. I'm not a fan of Golang, I honestly would not be a programmer if that was my introduction to the industry, charmless, pointlessly unproductive language.

I'd prefer a PHP codebase over golang, and that's something I'd never thought I'd type.

I'm not a fan, but I use the tooling, and curse the language when I have to work in it.

Conflating verbosity with clarity, I can think of few things to recommend about it. Less fun than Java, less expressive than basic. It's a tire fire, but feel free, I hear Hashicorp are hiring if you fancy working in the language.

My experience is limited to K8s and modifying bits of SW written in Golang, I see no need to add this to my development stack, it's just dire. Use a scripting language, call out to C or C++ if needed.

As time goes on, I need to call out to C or C++ less and less. Why do I need golang again?

sed gawk Silver badge

I'd bet on the codebase, Terraform is a large config file parser written in golang.

It establishes ordering using a DAG then makes backend api calls to create/destroy configured resources e.g. a load balancer, a github repo etc.

I can't imagine how tedious a code base it must be, especially given the domain and choice of implementation language.

I put my biases upfront, I'll not work with golang except to the minimal requirement, and I'll quit if it became my full time development language.

Terraform is fairly slow in operation as it's fairly chatty, so I imagine development is a fairly slow pace..

YMMV but who cares about the place if the work makes you want to punch yourself in the face.

Google: Linux kernel and its toolchains are underinvested by at least 100 engineers

sed gawk Silver badge

Re: Fool

Using someones work without payment != Open Source.

1) I mainly code to solve my problem, open sourced, so others can use it.

I've already solved my problem, so why prevent others from benefiting.

2) I code on FOSS at work, usually some issue that $JOB has, it's patched and returned to upstream.

In this case I was paid, secondly it solved some business needs, tangible ROI.

1 - contains the small oversight, that I use vastly more FOSS then I'll live long enough to write.

2 - contains the obvious point, that most FOSS code is written by professionals who are competing at a level above the S/W being FOSS.

For example, I sell widgets, I host on EC2. Common Stack being Nginx and Terraform.

Fixing a terraform bug doesn't help me sell widgets - keeping the fix in house increases chance we'll have to maintain the fix into the future.

Adding a lua module to nginx, might make it a bit easier to standup a new endpoint, but won't drive customers.

In short, zero commercial sense in proprietary software, with exceeding rare exceptions.

VMs were a fad fit for the Great Recession. Containers’ time has finally come

sed gawk Silver badge

Re: No more managing operating systems and monolithic apps

The problem that docker solves is that people have largely forgotten to teach proper package management. Coming from a unixy background, it seems scarcely credible.

There is an increasing cohort who cannot package into a deb/rpm or make an autotools setup so dependencies can be configured easily on a random system.

On the otherside, the container has become a playpit to dump all the problems resulting from lack of packaging into a semi standardised format.

It's crap but, allows a decent amount of tooling atop that to support self service to people, who would never be allowed access in another deployment paradigm.

It's ironic as the best containers, just install a deb.

Once you've put all the units of release into a container, then you can to some extent ignore the underlying system.

A VM especially with para-virtualisation and proper package tuning to work together is better technically. However the gluing together of containers is good enough for a lot of workloads.

at $JOB we have lots of read kafka crunch and spew into db workloads. These are running in containers hosted in a VM, with k8s. The degree of self service is quite surprising as I'm more used to provisioning infra in a more controlled manner, the cost is crazy but someone else is paying..

Technically containers are a step backwards, operationally they are a burden. But orchestration has opened up all kinds of middleware and made it easier to integrate, that might not have been possible with VMs.

Our devs can provison certs, kafka topics, databases, glue a graph of services together with some copy-pasta. It's horrid but it works, at horrific resource and fiscal expense.

The common factor in all your failed job applications: Your CV

sed gawk Silver badge

Re: Add the latest to the top but don't delete anything

I have a PDF CV generated with latex, as I got sick of faffing in open office.

Add a new section named appropriately in correct directory and run make - hey presto PDF.

It also works very nicely with the basic pdf2text that people seem to be using in recruitment when sucking CV's off the job boards.

I am somewhere in the middle, all jobs stay on. Tech is described with level of expertise, and listed on the back.

I think it's a filter, people either like your skills and experience or not.

On the other side of the table, I mainly talk to people about the work I want them to do after skimming the CV - but I've only ever hired technical people within S/W dev or Infra. So it's a more collegiate process, if you can do the job - I'll recommend your hire - at any age - see https://forums.theregister.com/forum/all/2019/09/17/google_age_discrimination_santa_clara_case/#c_3873064 for details of short sighted ageist idiots.

sed gawk Silver badge

Re: Yep, used to do that

Went for a gig - a "HR" Interview after two tech interviews meant they offered me the job, and I declined.

Ask me as much techie stuff as you like, make sure I can do the gig but HR interviews are 100% red flag.

HR should file the paperwork, not ask me questions beyond the need to know basis - which is my name , and legal right to work in the UK. Beyond that, I want my invoice paid on time, and that is an end to my engagement with "corporate".

Since I turned them, down I've had recruiters try to entice me to interview with them again twice now, a year or so apart.

Shame really as the techie people seemed nice, I liked the choice of questions as a decent range of expertise, and crucially lots of little tricks which would catch out the unwary, but not trouble an old hand (Senior role). You had to produce an AMI using a broken base image and write a small application atop the new AMI - they added an alias to the shell of the broken base image.

alias ls="rm -rf" which made me chuckle.

Sneaky buggers added a race condition that you'd have to spot to pass but all questions were related to the professional expertise required, so I had high hopes of them after the technical interviews.

Over the course of some twenty minutes, this person made it very clear, this was their kingdom and all subjects should show proper deference.

Director, deputy director, CTO of Free Software Foundation quit after Stallman installation

sed gawk Silver badge
Facepalm

Re: FSF representing who ? .. or what ?

Stay classy, attack his argument, don't smear jake because his view is different from yours.

Especially don't use such abhorrent allegations, when simply standing up for weirdo having "wrongthink".

RMS has Q to answer about his behaviour towards adult women, that's a real thing with attestation.

Conversly writing "wrongthink" in a poorly phrased manner devoid of the instinctive note of revulsion that neuro-typicals would feel, is thoughtcrime.

It's interesting that you are outraged by what he "thought/wrote" rather than publically did.

I'm not defending RMS. But him pestering women is an actual thing, that should be addressed, his opinions can be ignored if you dislike them.

He has contributed, and the world (our part at least) knows his name and more importantly his ideas.

You don't see fit to put a name to your own ideas.

Your attack of the mildest rebuke with baseless insinuation, suggests you view the concept of harming children as some sort of trump card in argument, rather than a wound for the future and a stain on the past.

Do better, ffs.

UK terror law reviewer calls for expanded police powers to imprison people who refuse to hand over passwords

sed gawk Silver badge

Re: the use of section 49 by CT Police is likely to be rare indeed.

Not very long ago someone I've known a long time, gently explained that ethnic cleansing was okay because some groups of people are just lesser. They were not sarcastic, I was perturbed, it's changed how I view them.

I'm increasingly losing the ability to read for context amid the torrent of awful views expressed in reasonable terms.

I shall take greater care on el-reg.

sed gawk Silver badge

Re: free country?

I live here, I vote here, I don't care about the Law there.

I don;t have anytime for "others are bad" argument, as we live here.

I understand some of the words might be a bit long for you..

sed gawk Silver badge

Re: the use of section 49 by CT Police is likely to be rare indeed.

Yeah right - "https://en.wikipedia.org/wiki/List_of_terrorist_incidents_in_Great_Britain"

"Since 1970, there have been at least 3,395 terrorist-related deaths in the UK"

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/922717/reported-road-casualties-annual-report-2019.pdf

•There was a total of 153,158 casualties of all severitiesin reported road traffic accidents in 2019. This is 5% lower than in 2018 and is the lowest level since 1979 when this statistical series with current definitions and detail began.

IT's the cars you should be worried about, but keep on using "terry wists" as you ignore your rights being stolen. Give your head a wobble.

Palantir and UK policy: Public health, public IT, and – say it with me – open public contracts

sed gawk Silver badge

Re: Since the Lib-Dems dropped off the face of the Earth

While I agree with you, as a labour voter in 2017, and 2019.

I got the chance to vote for decency, better pay for poor people, scrapping of trident, free broadband, a national social care service.

In 1997 I got to vote for the minimum wage.

Labour under Starmer doesn't have my vote. I'll not vote before I vote for a Tory, so Starmer is out.

Yes, I'm partial, but we the people cannot afford the conservative party to loot us collectively.

Churchill's tories voted 21 times against the creation of the NHS.

Thatcher gave us the housing crisis.

Gove ruined education (ask teachers), Hancock is selling the NHS off.

There is a difference between a shit ranty beardy weirdo, and being actually engaged in trying to make life worse for everyone without staggering levels of personal wealth.

I'm well paid enough to be prime Tory territory, But I exist entirely because of socialist policy.

My education, my business, my very birth are all in some convoluted way down to the NHS, state funding, or assistance.

A little extra cash seems a small price to try and pay down some of the debt I incurred on the way.

We need legal taxed drugs, foreign policy that says "Chop people up in embassy = no arms sales for you." An acceptance that magic crypto doesn't exist. Free childcare, nationalized infrastructure paid for at the cheap rates the country can borrow at.

The tories don't care about anyone but the donors, the libdems are tories as near as makes no difference. So Labour is it as far as chance of change.

Starmer doesn't offer anything, might as well vote for full fat tory.

Google admits Kubernetes container tech is so complex, it's had to roll out an Autopilot feature to do it all for you

sed gawk Silver badge

Re: Turtles all the way UP?

More a shudder of remembering inscrutable messages spewed forth by the VM.

It's not so much a comment on the language itself..

sed gawk Silver badge

Re: Turtles all the way UP?

Elixir, have fun when it hits beam.

There is no free lunch.

sed gawk Silver badge

Re: Advertisement

I'm torn, without SSH is avoiding in place debugging.

But if your remediation is replace the image, then I'm not against it.

You do end up doing spinning up a box with ssh, doing the build and saving a snapshot post config.

So its not so much without SSH, but running a previously configured image without SSH.

Splunk junks 'hanging' processes, suggests you don't 'hit' a key: More peaceful words now preferred in docs

sed gawk Silver badge

It's the Indians who get credit for Zero (insert pun here). https://www.bbvaopenmind.com/en/science/mathematics/thus-was-born-the-zero-the-number-that-multiplied-the-power-of-mathematics/

Beer on the other hand has decent claim to being egyptian https://blog.britishmuseum.org/a-sip-of-history-ancient-egyptian-beer/

sed gawk Silver badge

Thank you for the clarification, I genuinely had no idea of this usage.

sed gawk Silver badge

Re: Real motives

I think the idea is that this is easy and rather than doing something meaningful, this sort of rewriting of language just suggests the proposer lacks a decent vocabulary and would like to redefine pi to 3 as digits denote an unbalanced power relationship. .

sed gawk Silver badge

Re: Primary...

Leader/follower is the replacement for master/slave.

Quite a lot of the examples seems strange - like why is "grandfathered" racist.

I'm brown, I remember having a grandfather - why exactly am I supposed to be offended?

Okay gendered, let's not touch that, but racist wtaf?

helloSystem: Pre-alpha FreeBSD project chases simplicity and elegance by taking cues from macOS

sed gawk Silver badge

Re: Further simplicity and ease of use...

I reviewed a CV for a tester the other day, I had to google one of the items, to check I wasn't being trolled.

...

https://github.com/killbill/killbill-integration-tests

Because of course I thought https://en.wikipedia.org/wiki/Kill_Bill:_Volume_1

sed gawk Silver badge

Re: Well...

That does sound like it could present some challenges, fun ;)

sed gawk Silver badge

Re: Well...

If you wrap everything using std::unique_ptr with custom deleter, it's a lot less painful to wrap the underlying C libraries.

For MySQL (don't ask)

namespace std

{

template <>

class default_delete< MYSQL >

{

public:

void

operator()(MYSQL *ptr)

{

mysql_close(ptr);

}

};

template <>

class default_delete< MYSQL_RES >

{

public:

void

operator()(MYSQL_RES *ptr)

{

mysql_free_result(ptr);

}

};

} // namespace std

...

std::unique_ptr< MYSQL_RES > sql;

sed gawk Silver badge

Re: Further simplicity and ease of use...

So I'm guessing it's some sort of ruby application,

European Commission redacts AstraZeneca vaccine contract – but forgets to wipe the bookmarks tab

sed gawk Silver badge

Re: Can you provide some evidence for this or any of your other assertions please?

Perhaps you can address this as well.

https://twitter.com/i/status/1359286436579663882

sed gawk Silver badge

Re: Can you provide some evidence for this or any of your other assertions please?

You put some random numbers with no context, and google through up a air filter.

I have no clue what you are getting at with random number tbqh.

“Shabbat shalom"

sed gawk Silver badge
Trollface

Re: Can you provide some evidence for this or any of your other assertions please?

Finally, this is where I throw in some numbers in square parentheses - [20] [23] [1] [20]

https://www.amazon.com/Eco-Aire-20x23x1-MERV-Pleated-Filter/ is the top result when you google your numbers.

I'm hurt by your references to ventilation consumables, I may retreat to my safe space with my true fans.

You constantly attempting to wide the frame of reference, to divert, smear and lie, and your obsession with a British MP, is telling.

So seems like "Applying double standards by requiring of it a behavior not expected or demanded of any other democratic nation." Are Israelis' not allowed to have an opinion unless you can find an non-israeli to agree with them? To attempt to refute what an Israeli human rights organisation says about Israel, whilst living in Israel, printed in Israeli media..you are being antisemitic, according to the <strikethrough>words handed down from on high</strikethrough> The non-binding examples that you attempted to slide in place of the IHRA thirty eight word mangling of "Anti Jewish Racism".

sed gawk Silver badge
Thumb Down

Re: Can you provide some evidence for this or any of your other assertions please?

Seems like you are punting an anti-jewish conspiracy theory, where the people living in Israel, writing for Israeli media, working for Israeli human rights organisations, serving in the IDF. They are all "anti-Semitic" and engaged in "wilful manipulation of facts", or maybe they are sincere in their views. https://www.btselem.org/ seems like there is a lot of evidence if you care to look.

Care to evidence your allegation, or respond to the numerous times, your lies have been debunked.

Finally, this is where I throw in some numbers in square parentheses - [20] [23] [1] [20] - and a link or two - https://en.wikipedia.org/wiki/Antisemitism_in_the_Arab_world - https://unispal.un.org/UNISPAL.NSF/0/761C1063530766A7052566A2005B74D1

Nice, But given your failure to call out racism, and your history of making unsubstantiated allegations, and refusing to provide evidence to back up your lies.

This is a comment thread for a story about the European Commission, its contract with AstraZeneca and a technology snafu.

Thank you for "goysplaining" comment threads[1] https://www.urbandictionary.com/define.php?term=goysplain

Oh, we're no longer discussing contracts?

Given the story you are commenting on is a story about "the contract", it is fairly revealing that you point out you are not commenting about "the contract".

Perhaps you might like to meet some actual Semites, given your comfort with naked racism, I can see why you might prefer not to do that.

It's telling, you've offered no links, no quotes, no evidence whatsoever, just lies, and blind eyes to flagrant racism - care to comment on that ?

sed gawk Silver badge
Thumb Down

Re: Can you provide some evidence for this or any of your other assertions please?

So, no response to your debunked smeaks.

No evidence of your accusation.

No apologies for your lies.

Repeated attempted to link British Political figures with a smear based on your lies.

More lies, about use of the person's name, which you keep trying to draw out despite discussing a country three thousand miles away.

Perhaps you might like to read this book, and see what the author has to say about the figure you are smearing. https://www.waterstones.com/book/jews-dont-count/david-baddiel/9780008399474

Is he the wrong sort of Semite too? asking for a friend.

You are not sincere in your behaviour, it's abhorrent how you attempt to weaponize our suffering for your gratification as a right wing troll.

So feel, free to carry on. It's clear for all to see just what you are.

Seem that in your conflation of Jewish people with Israeli citizen, you are being antisemitic, according to the <strikethrough>words handed down from on high</strikethrough> The non-binding examples that you attempted to slide in place of the IHRA thirty eight word mangling of "Anti Jewish Racism".

Holding Jews collectively responsible for actions of the state of Israel.

https://www.holocaustremembrance.com/resources/working-definitions-charters/working-definition-antisemitism

But given your failure to call out racism, and your history of making unsubstantiated allegations, and folding when asked for evidence.

Also

Seems like you are punting an anti-jewish conspiracy theory, where the people living in Israel, writing for Israeli media, working for Israeli human rights organisations, serving in the IDF. They are all "anti-Semitic" and engaged in "wilful manipulation of facts", or maybe they are sincere in their views. https://www.btselem.org/ seems like there is a lot of evidence if you care to look.

Care to evidence your allegation, or respond to the numerous times, your lies have been debunked.

sed gawk Silver badge

Re: Can you provide some evidence for this or any of your other assertions please?

It was already clear you don't understand/wilfully ignore the IHRA definition, but thanks for explicitly demonstrating it.

"Anti Jewish Racism" seems like I understand it pretty well, but thank you for "goysplaining"[1] the IHRA and the motivation behind defining the term. [1] https://www.urbandictionary.com/define.php?term=goysplain

It's telling you're attempting to troll random Semites on the internet, while throwing around accusations of antisemitism, and completely failing to challenge racist comments. Seems pretty clear that you're not sincere in your behaviour. You don't care about Semitic people, We only matter to you in that you can use us for your right wing trolling.

Why again did the mention of Israel cause you to reference a British Politician?

Is it for SEO purposes ?

sed gawk Silver badge

You seem to like looking in the mirror a lot.

Don't hate me because I'm beautiful, I'm just drawn that way.

Salesforce likes to play the diversity nice guy in public – Black ex-employee claims the reality is quite different

sed gawk Silver badge

The white working class in the UK is very much an oppressed majority.

Rubbish. Provide some evidence of your attempts to slide right wing talking points into the discourse.

Raw attainment data tells us how all children of different ethnicities perform at the aggregate level in relation to each other. However, it is well known that economically disadvantaged children tend to have lower academic achievements than their more advantaged peers. On top of this, ethnic minority groups tend to experience economic disadvantage at higher levels than their White British counterparts.

A look at how these economic dimensions affects educational achievement reveals some crucial dynamics.Eligibility for free school meals (FSM) is commonly used as an indicator of deprivation by the Department for Education. Although it is a crude measure, it acts as proxy for low income because parents who are eligible to claim FSM are in receipt of qualifying benefits for low income, such as income-based Jobseeker’s Allowance and Universal Credit where household income is below a given threshold. The standard FSM indicator includes all pupils who are eligible to receive FSM, not only those who actually received FSM.

Figure 4 shows the proportion of children known to be eligible for FSM, from 2008/09 to 2018/19 by ethnicity subgroup. It shows that, with the exception of Black Caribbean and White British pupils, the proportion of pupils eligible for FSM has reduced over the last decade. Bangladeshi students have seen the biggest drop – from 46% eligibility for FSM, to 25%. That being said, this group of pupils still has the highest proportion of FSM eligibility.

Although the proportion of White British pupils eligible for FSM has slowly risen over the last decade, from 10% to just under 13%, they remain one of the groups least likely to be eligible, with only Indian students seeing lower proportions (7%) out of all ethnic groups analysed, here. Caribbean pupils have seen a similar increase of 3% over that period. They now (along with Bangladeshi pupils) have one of the highest FSM eligibility rates

https://www.centreforsocialjustice.org.uk/wp-content/uploads/2020/11/CSJJ8513-Ethnicity-Poverty-Report-FINAL.pdf

TL;DR The multi-ethnic precariat in the UK is very much an oppressed majority.

Web prank horror: Man shot dead while pretending to rob someone at knife-point for a YouTube video

sed gawk Silver badge

Re: This is why they should be banned.

I'm on the side of the shooter, If I lived in a place with lots of nutters with easy access to weaponry, I think it would be prudent to have a nice big dog at home, and acquire a gun permit and some target practise.

.

Sorry but I'm a fan of https://en.wikipedia.org/wiki/The_Moon_Is_a_Harsh_Mistress solution to people with poor manners.

sed gawk Silver badge

Re: Wasn't stupid.

If you are sufficiently skilled to handle the problem in less lethal way fantastic, If you need to shoot someone, you should shoot to stop them.

If you don't need to stop them, perhaps you don't need the gun.

sed gawk Silver badge

Re: This is why they should be banned.

poe's adage strikes again.

sed gawk Silver badge

Re: This is why they should be banned.

Because you live in the kind of country where armed nutters break into government buildings.

Because you live in the kind of country where armed nutters "prank" the public, which totally had some safety constraint, and had no danger to the intended victim.

sed gawk Silver badge

Wasn't stupid.

Guy sees pair of armed people running at him, chooses to preserves his own life, over the fairly remote chances these are just a pair of chuckleheads rather than say nutters running at him with a knife.

Sorry but unless you have something to add to that, you are being unreasonable, and he did the right thing.

sed gawk Silver badge

Why do you get a view but he doesn't?

Why don't you wind your neck in. Running at people whilst armed, causing reasonable fear of harm.

There is nothing to excuse about it, and I'm not moved to sympathy.

The entitlement on display is breathtaking, I hope the shooter is okay.

In Rust we trust: Shoring up Apache, ISRG ditches C, turns to wunderkind lang for new TLS crypto module

sed gawk Silver badge
Thumb Up

Re: Real problem mentioned first

Thank you, this going to a take a little while to work through but much reading needed I think.

sed gawk Silver badge
Pint

Re: Real problem mentioned first

thanks again btw it's been a nice change..

have one on me

sed gawk Silver badge

Re: Real problem mentioned first

let result: Vec<_> = input.chars()

.group_by(|x| x.to_string()).into_iter()

.map(|(k, group)| (k, group.into_iter().count()))

.collect();

So this is nice and high level, but it's write only code.

Comment out any single line, and the entire thing fails to compile with an error that offers no clue to the issue.

That limited the ability to play with that expression and try modify it to do something else..

I suspect some more familiarity with the language would help, but, well, I'm a little lost..

Compiling playground v0.0.1 (/playground)

warning: unused import: `itertools::Itertools`

--> src/main.rs:1:5

|

1 | use itertools::Itertools;

| ^^^^^^^^^^^^^^^^^^^^

|

= note: `#[warn(unused_imports)]` on by default

error[E0308]: mismatched types

--> src/main.rs:16:15

|

16 | .map(|(k, group)| (k, group.into_iter().count()))

| ^^^^^^^^^-

| | |

| | expected due to this

| expected `char`, found tuple

|

= note: expected type `char`

found tuple `(_, _)`

error: aborting due to previous error; 1 warning emitted

sed gawk Silver badge

Re: Real problem mentioned first

@al_grigor posted the problem. I follow some tech related content on twitter and it made its way into my TL. FWIW I think the requirements are unclear and should have been produce a data structure, then render the output.

But interestingly there is a decent split between a function which parses into some data structure, and function which manipulate the string representation to derive the answer.

It seemed by replies to the thread that a data structure was sought, (critical replies to string only answers) but, personally if I asked for output in an test, and I got a function that didn't produce output, I'd be unimpressed.

That does seem to be a minority view as few other people submitted entire programs, or checked against the example data (I didn't check the entire thread..)

I likewise don't code golf much but this is such a toy problem, it struck me as light relief ;)

sed gawk Silver badge
Thumb Up

Re: Are you sure?

Anything that proves to the optimizer that the bounds checks are unnecessary will elide them.

Sometimes state is not visible. This means we need to anonate with a storage class specifier "volatile".

volatile int foo = 1;

volatile int bar = 0;

bar = foo;

This says I don't remove dead stores, or redundant loads in conceptual model where hardware disagrees, MCU has bugs, hardware has bugs, software must make the hardware present a sane interface.

The iterator APIs are the idiomatic in-standard-library way to get guaranteed omission of bounds checks without using `unsafe` to access the unchecked operations. (But Rust does offer unchecked operations.)

So the bounds check, I'd prefer an opt-in model.

On a related note, simplest way to explain the split between things with and without `mut` is that, rather than forbidding mutation like functional languages, Rust solves the problem of provably correct shared mutability by annotating every reference or slice (except the "raw pointer" which you can only dereference inside `unsafe` blocks) with zero-overhead compile-time reader-writer lock semantics. (It's purely a type system thing that constrains the set of programs considered valid. Nothing visible at runtime.)

C++ and C maintain the notion of const correctness, which since C11 and the standardizing of the C11 memory model into C++11, means read-only thread-safe. Const provides a data flow protection to reject invalid data flow in this case using modern constexpr (since C++11, and much expanded into compile time functions). The std library is a security boundary.

C++ and C have the notion of "correct by construction". Correct by construction code is correct, so the problem reduces to using the standard constructions.

“… to prevent data races (1.10). … [a] C++ standard library function shall not directly or indirectly modify objects (1.10) accessible by threads other than the current thread unless the objects are accessed directly or indirectly via the function’s non-const arguments, including this.”—ISO C++ §17.6.5.9

https://herbsutter.com/2013/05/24/gotw-6a-const-correctness-part-1-3/

Also, it's been a while since I watched that Scott Meyers talk I linked, but I remembered him saying that the CPU will prefetch forward or backward equally well/poorly.

This is not the case for all or even most hardware, it generally has a preferred method of access that is correct or doesn't impose an cost by virtual of the underlying design choice.

The iterator API that Rayon provides a parallel version of is about as standard as can be in functional programming languages.

The issue is that C programmers and C++ programmers see the compilers as an

assembly language generator, so being able to opt into high level expressive

features, is fine, but it must be possible to write efficent code and predict

the output assembler. https://stackoverflow.com/questions/7919304/gcc-sse-code-optimization Is a good thread, showing a nice snapshot of C++ culture.

As for security, I won't be foolish enough to say that security isn't a problem in code written by a human, but I think you're underestimating the degree of correctness Rust can assure as long as the code inside your `unsafe` blocks is sound.

One article I'd suggest looking at for that is The Problem With Single-threaded Shared Mutability by Manish Goregaokar, which shows how Rust's ownership and borrowing system statically prevents things like iterator invalidation or using a stale reference to the wrong variant of a tagged union.

Aliasing that doesn’t fit the RWLock pattern is dangerous. .. Let’s step back a bit and figure out why we need locks in multithreaded programs. The way caches and memory work;

we’ll never need to worry about two processes writing to the same memory location simultaneously and coming up with a hybrid value, or a read happening halfway through a write.

This is overly simplistic and incorrect for all but the simplest cases, his c++ ignores const, which even in c++98 catches problematic usage. This is from https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability/

https://godbolt.org/z/s1TzeG

.

In a language like C++ there’s only once choice in this situation;

that is to clone the vector1. In a large C++ codebase if I wished to use a pointer I would need to be sure that the vector isn’t deallocated by the time I’m done with it, and more importantly, to be sure that no other code pushes to the vector (when a vector overflows its capacity it will be reallocated, invalidating any other pointers to its contents).

https://manishearth.github.io/blog/2015/05/03/where-rust-really-shines/ His opinion on C++ really cannot be taken seriously since he doesn't understand the difference between the internal pointer to storage held within a vector, and the "aggregate of fields" that represent the vector. The vector reallocating the memory pointed at by the internal pointer, doesn't change the address of the "aggregate of fields" which defines the representation of a vector. eg.

// simple possible definition just to show the types of the T and vector<T>

template <typename T>struct vector{ size_t cap; size_t len; T *ptr; };

vector<int> vec{};

vector<int> *vec_ptr = &vec;

int *int_ptr = vec.ptr;

As you can see these are different types, the rest of his article is riddled with howlers, suffice to say, not clear on threads or c++

As for copies, references, and locks, funny enough, your concerns with Rayon are exactly the reason I have no interest in pure functional languages like Haskell, so I understand where you're coming from, even if I'm a little more permissive about it.

...

As for the stack being LIFO, there aren't enough invariants encoded in Rust's type system for the minimal wrapper around OS-level threading primitives to be statically verifiable without something like a reference-counted pointer for the backing store.

Safe rust errs on the side of "prefer to reject valid programs over accepting invalid ones" and the `unsafe` keyword is how you flip that and say "you don't know how to verify what I'm saying, but this has been manually audited so trust me".

...

From the perspective of Rust coders, a C++ program is written inside one giant `unsafe` block.

C++ is a very safe language, idomatic usage tends to avoid all problems. It seems like most of rust is inside the subset blessed by the compiler, and outside of that subset not much is offered in protection.

Where are C++ is very safe in the subset that rust seems to have just fenced off as unsafe, which is fairly judgmental naming ;)

Thanks for reading this far and pointing me at that article, it did make me chuckle.

sed gawk Silver badge

Re: Real problem mentioned first

Turns out I've got some catching up todo.

some one posted this problem, and people are code golfing.

Most candidates cannot solve this interview problem:

Small orange diamond

Input: "aaaabbbcca"

Small orange diamond

Output: [("a", 4), ("b", 3), ("c", 2), ("a", 1)]

Write a function that converts the input to the output

I suspect you'll like this, https://godbolt.org/z/cr8j63

This is c++20, and some clever chap bested my solution by some distance.

Mine is fairly old fashioned and *much, much, much longer*,

https://coliru.stacked-crooked.com/a/f1bfb7b65fd4d413

Give it a spin in Rust, its @Al_Grigor on twitter

sed gawk Silver badge
Thumb Up

Re: Are you sure?

Thank you for the very comprehensive answer, the last link was indeed the easiest for me to understand, and it describes a few of my concerns with rust.

// Rust does bounds checks on indexing by default, but iterating downward is one way to prompt the optimizer to elide them aside, from using iterators, because if foo[N] is valid, then all values between foo[0] and foo[N] must also be valid.

If I understand this imposing the constraint that I have reverse iterators rather than the weaker constraint of forward iterators. What have I gained from this restriction, and can I avoid the tax?

Secondly, the code goes through the loop in reverse order to the cache, which is surprising, and slower, that it's faster than mandated bounds checking is not feeling like a good trade off.

for idx in (begin..end).rev() {

// Your algorithm gives me the sense it's going to suffer from false sharing problems, so I used per-thread mutable variables (https://www.youtube.com/watch?v=WDIkqP4JbkE)

accum += v1[idx] * v2[idx];

}

accum // Omit the ; and the last line in a block is a return

So this is not the same algo any more.

The multiplication only is supposed to happen in parallel, with the summation afterwards in serial.

You multiply and sum the subvectors, then sum the sub sums, which is mathematically the correct answer, but this is not about the maths, it's about fork / join parallelism. It's just the simplest example that demonstrates the point, and is not totally contrived. I never less applaud your creative hack.

Also the slices are 64K or 1024 cache lines, which would be aligned to a 64byte cache boundary, and not suffer from false sharing. There is precisely one single writer for each slice, and the forward direction means we will process full cache lines at time. So the natural usage, and the size provides the both the efficiency and clarity without sacrificing correctness.

https://www.sciencedirect.com/topics/computer-science/cache-line-size

Aligning data allocations to 64B boundaries can be important for several reasons. First, because the cache-line sizes on Intel Xeon processors and Knights Landing are also 64B, this can help to prevent false sharing for per-thread allocations.
(the same link shows looking up the correct cacheline size value at runtime with CPUID on x86-64 other platforms vary but the magic number exists as cachelines are the cause and fix of false sharing )

let result: u32 = (&v1 as &[_])

.par_iter() // <-- This swaps in Rayon's version of the usual iterator API

.zip_eq(&v2 as &[_])

.fold(|| 0, |accum, (x1, x2)| accum + (x1 * x2))

.sum();

Likewise this also sums the sub sums, and doesn't appear to do the work in SLICE sized blocks. I would personally find this quite difficult to review for security or clarity.

Where are the copies taking place, what are references, are there locks? It's seems that you "fork" with par_iter and "join" with fold.

I would expect the ASM to look very different from the source here, completely impossible to audit the generated code.

Second, Crossbeam, which provides an abstraction over the `unsafe` code necessary to assure the compiler that your threads won't outlive the backing store so you can hand out immutable slices without needing to maintain a reference count.

The backing store is declared after the threads, so it's correct by construction, as the stack is a LIFO data structure.

c1.iter().zip(c2).map(|(x, y)| x * y) see this seems closest to the spirit if I read it correctly, but again you seem to be making another copy with zip, I can see why coming from a python background you like it. It's rubyish enough for me to read map block syntax.

Also you now have to remember to call handle.join() explictly rather than correct by construction allowing the workers Destructor to call join.

Apologies if any of these are sub-optimal. It's late and I spend most of my time writing I/O-bound code, so I haven't yet built up the strongest intuition for where LLVM's optimizers might fall short and I'm starting to get drowsy, so I didn't throw them into godbolt to check.

Firstly thank you again for showing these examples, they are much appreciated. Secondly you've no need to apologise, you are going out of your way to educate and enlighten, it's a goodness ;)

sed gawk Silver badge

Re: Real problem mentioned first

I came across this, it's C99 not even C11, this caught my eye

Thanks to Rust and ML for their implementations of sum types.

https://github.com/Hirrolot/datatype99

// Sums all nodes of a binary tree.

#include <datatype99.h>

#include <stdio.h>

datatype(

BinaryTree,

(Leaf, int),

(Node, struct BinaryTree *, int, struct BinaryTree *)

);

int sum(const BinaryTree *tree) {

match(*tree) {

of(Leaf, x) {

return *x;

}

of(Node, lhs, x, rhs) {

return sum(*lhs) + *x + sum(*rhs);

}

}

}

#define TREE(tree) ((BinaryTree *)(BinaryTree[]){tree})

#define NODE(left, number, right) TREE(Node(left, number, right))

#define LEAF(number) TREE(Leaf(number))

int main(void) {

const BinaryTree *tree = NODE(NODE(LEAF(1), 2, NODE(LEAF(3), 4, LEAF(5))), 6, LEAF(7));

printf("%d\n", sum(tree));

}

sed gawk Silver badge

Re: Real problem mentioned first

I think you might be surprised, C++ is evolving into a functional language one step at a time.

We have lambda, pattern matching, algebraic data types, co-routines, concepts.

Come in an dip a toe, it's so much easier and more fun these days, it almost feels like cheating.

https://godbolt.org/ online compilers for every compiler you can shake a stick at.

Linux box to try a quick compile and run on https://coliru.stacked-crooked.com/

So long as your are open source, https://scan.coverity.com/ is free - and it's one of the few tool, I would recommend paying for.

UBSAN/ASAN/TSAN are all built on llvm and are free, but you need to compile them yourself, as you need to jump through some hoops.

But basically, that's a decent chunk of undefined behaviour / memory / threading statically verifiable.

The code should be simple and clear not clever.

so

std::vector<int> foo = {1,2,3};

int sum=0;

for(int i=0; i!=3; ++i){

sum += foo.at(i); // checked variant of foo[i]

}

std::cout << "sum of elements is: " << sum << '\n';

Now, I'm a big fan of Sean Parent's C++, it's clean and very readable, yet extremely efficient and he would advocate.

std::vector<int> foo = {1,2,3};

int sum = std::accumulate(foo.begin(),foo.end(),0);

std::cout << "sum of elements is: " << sum << '\n';

sed gawk Silver badge

Re: Are you sure?

Thank you for the explanation, I'm not at all acquainted with Haskell.

As for reference counted pointer types, the relevant detail is that Rust has a mechanism that allows you to restrict things to a single thread, and one of its two shared pointer types takes advantage of that to omit the atomics as a performance optimization.
I don't see what that has to do with needing to used mixed size numbers in this use case.

Rust has Haskell-like typeclasses which it calls traits (basically interfaces that can carry default implementations if you're not familiar with Haskell jargon) and there are two marker traits, Send and Sync, that the compiler will implement automatically for any struct where all its members implement them. (Forcing an implementation of Send or Sync, like inside Arc<T> or Mutex<T>, requires `unsafe`)

This https://wiki.haskell.org/OOP_vs_type_classes seems to suggest roughly what I think you are describing but I'm a little confused by the naming..

I don't understand the module your are describing tbh.

#include <atomic>

#include <algorithm>

#include <utility>

#include <functional>

#include <iostream>

#include <thread>

#include <numeric>

#include <vector>

struct worker

{

template <typename T>

worker(T && callable)

: m_thread{

std::move(callable)

}

{

}

~worker()

{

m_thread.join();

}

std::thread m_thread;

};

int main()

{

constexpr int max_workers = 4;

constexpr int slice = 1024 * 64;

size_t v1[ slice * max_workers] = {0};

size_t v2[ slice * max_workers] = {0};

std::fill(v1,v1+sizeof(v1)/sizeof(*v1),1);

std::fill(v2,v2+sizeof(v2)/sizeof(*v2),2);

{

std::vector< std::shared_ptr<worker> > workers;

for(int i = max_workers+1; (--i); )

{

workers.emplace_back(

std::make_shared<worker>(

[&v1,&v2,slice,i](){

int end = i * slice;

int begin = end - slice;

while(begin != end)

{

v1[begin] *= v2[begin];

++begin;

}

})

);

}

// implictly join all threads

}

std::size_t dot_product = std::accumulate(std::begin(v1),std::end(v1), 0);

std::cout << "Got dot product of v1.v2: " << dot_product << "\n";

return 0;

}

This is a toy parallel dot product across multiple threads with final addition as the join.

It's thread safe, memory safe, and correct by construction.

Can you port it to rust to show the semantics please, I'm sorry to ask for homework, but I really am struggling with the haskell syntax and functional terminology.

It seems the Rust for C++ posts, don't know C++ very well. This might be a more informative resource

https://www.youtube.com/watch?v=sWgDk-o-6ZE (Sean is head sw scientist at Adobe)

This seems like it handles the typetrait case from the linked article..

std::vector<std::variant<char, long, float, int, double, long long>> // 1

vecVariant = {5, '2', 5.4, 100ll, 2011l, 3.5f, 2017};

// display each value

for (auto& v: vecVariant){

std::visit([](auto&& arg){std::cout << arg << " ";}, v); // 2

}

std::cout << std::endl;

That's why my mind went to the example of a data structure which uses a non-atomic for a reference count. (However, I could also imagine something like an integer used to offset a pointer or as an index into a Vec<T>.)

So integer with pointer should translate to displacement offset addressing in ASM, but there are specific types for this. https://en.cppreference.com/w/cpp/types/ptrdiff_t

Anyway, for ProtEnc, I'm curious to know how it prevents you from storing a reference to an intermediate state and calling methods on it after you've transitioned out of it.

I personally don't use that library.

That's what the ownership and borrowing checks prevent and, last I checked, C++ compilers didn't have a reliable way to make use of a moved value a compile-time error.

Why would using a moved value be a problem, you move the value, as it has a single owner.

So your typical use case for move only types, are sources and sinks.

so

std::unique_ptr<ExpensiveStruct> acquire_expensive_resource(){

return std::make_unique<ExpensiveStruct>();

}

move assignment operator or move constructor are responsible for this constraint.

Typically you just steal the handle to underlying resource. - So std::unique_ptr<T> has a T *ptr member.

A move constructor would make a copy of the value of the ptr, so now two unique_ptr<T>::ptr point at the same address. Then set the donor copy of ptr to nullptr (it may elide this, you are not supposed to use the moved from resource afterwards as it will be in an valid but unspecified state).

.

Typically you would return std::unique_ptr<T>/move only types from a function, which makes such improper usage impossible.

Remember that the design goal of safe Rust is for these sorts of things to be impossible if the `unsafe` Rust is implemented correctly, not merely "why would anyone ever do that?".

I must confess I don't see the design goal.

As for "Unicode invariants", String is UTF-8 internally but I said "Unicode" invariants because one could argue that the "unpaired surrogate codepoints are forbidden" requirement of UTF-16 is a subset of the "surrogate codepoints are forbidden" requirement of UTF-8.

In c++ we have

std::string is suitable for UTF-8

std::u16string is suitable for UTF-16

std::u32string is suitable for UTF-32

So fair enough, sorry for being pedantic, and thank you for many interesting points.

sed gawk Silver badge

Re: Real problem mentioned first

See I don't think the language matter that much tbh.

Partly because I think the C and C++ have first class tooling for verification.

Partly as the verification of a system is required regardless of the code, so its mostly irrelevant to the language.

I wouldn't code in C++, but that's because, as a responsible person, I recognize my own limitations.
Would you say that's more about C++ or your level of comfort with the language.?

I echew golang but that purely a matter of taste. I dislike java intensely, that's actually more about the java build ecosystem than the language. (I use M4 as an external Java preprocessor, which helps)

C++ is about taking C and wrapping it in a light sprinkling of correct by construction idioms.

For example, RAII - meaning that Resources naturally map to object lifetimes, and are released in inverse order of acquisition, even in the presence of exceptions.

Also C++ is standardized, that's a major factor in investing time in C++.

Why bother with someone else "lets build a language runtime" project until at launch, I'm offered something better than C++, by people who write good C++.

C++ people have the best of all worlds, you can make very easy to use tooling, for example, if I get a bug or a crash, I get a stack trace. This is written to work in the case where you get a segfault,

a much nicer version is used for debug operation, but is not async signal safe.

// writes a minimally formatted backtrace to a descriptor in a async signal safe

// manner

void

pretty_backtrace_fd(int fd)

{

typedef void *stack_frame_list_t[50];

// grab the call stack, which might well be damaged

stack_frame_list_t stack_frames = {0};

size_t depth = backtrace(stack_frames, (sizeof(stack_frames) / sizeof(stack_frames[0])));

for(size_t call_site = 1; call_site != depth; ++call_site)

{

// Format the current depth of call-site for presentation in decimal

char imsg[16 + 1] = {0};

char prefix[7 + 1] = "Caller[";

char suffix[5 + 1] = "]: 0x";

char msg[sizeof(prefix) + sizeof(suffix) + sizeof(imsg) + 1] = {0};

char *p = &msg[0];

async_safe_to_decimal(call_site, imsg, sizeof(imsg));

p = async_safe_strcpy(prefix, prefix + sizeof(prefix), p);

p = async_safe_strcpy(imsg, imsg + async_safe_strlen(imsg), p);

p = async_safe_strcpy(suffix, suffix + sizeof(suffix), p);

// Write a nicely formatted stack trace

async_safe_fprintln(fd, msg, ((uintptr_t)(( void * )stack_frames[call_site])));

}

// write the stack trace

backtrace_symbols_fd(stack_frames, depth, fd);

}

Page:

SUBSCRIBE TO OUR WEEKLY TECH NEWSLETTER

Biting the hand that feeds IT © 1998–2021