EFF policy says bots can code but humans must write the docs

The Electronic Frontier Foundation says it will accept LLM generated code from contributors to its open source projects but will draw the line at non-human generated comments and documentation. Robot arm control the Steel sheet cutting process in the industrial factory Software engineer reveals the dirty little secret about …

[elsergiovolador]

LLM

> Now create documentation and comments. Make sure they look like human.

> I have generated the documentation and comments now, they look like human as requested.

O

\

/\

O

\

/\

[David 132]

Re: LLM

Another fine attempt at ASCII art stymied by the Reg comment system's insistence on inserting paragraph linebreaks everywhere?

[Paul Hovnanian]

Re: LLM

That has a basis in history. That human has been drawn and quartered.

[Pete 2]

Sic

>will draw the line at non-human generated comments and documentation

And while "LLMs excel at producing code that looks mostly human generated" they pair said these models often have "underlying bugs that can be replicated at scale.

Can we therefore draw a conclusion from this?

[O'Reg Inalsin]

Re: Sic

I read right over that with an internal spell correcter, so it really doesn't bother me. As for comments being AI generated, I wouldn't really care so long as they had been thoroughly read and checked for accuracy (of course), but also very importantly for brevity. Both in code and comments, AI can, and too frequently does, violate Occam's razor by many orders of magnitude, obscuring the gist completely, all while having perfect spelling and grammar.

[ChoHag]

Re: Sic

Spelling is nothing more than a conspiracy of publishers to sell dictionaries.

[Throg]

Worst of Both Worlds?

If there’s one thing that decades in the industry has taught me, it’s that we software engineers aren’t great at writing useful documentation.

So shoddy code AND terrible documentation?

Of course the best option is clean, readable code written by an experienced engineer. Minimal documentation required - mostly just the public APIs.

[Anonymous Coward]

Re: Worst of Both Worlds?

Nope ...

There is no excuse or 'magic' process that makes 'Documentation' no longer needed !!!

I really don't understand the problem with documentation !!!???

It is a useful way of understanding what you have produced and going through the logic again while you write the documentation.

It helps to highlight assumptions and errors in the original process AND most usefully it helps the Customer to use the 'thing' you produced.

Helping the customer use the product reduces the amount of 'Customer Support' needed and actually helps to sell more of the product !!!

Word of mouth sells the quality of the product and the quality of the support (because you needed so little of it !!!)

The more you create documentation the easier it gets ... until it is no longer the 'chore' you thought it was !!!

:)

[segfault188]

Re: Worst of Both Worlds?

When I began my first software development job (decades ago), the first task allocated to me was writing user documentation for software which hadn't yet been written.

Based on the customer requirements, this meant that we had to understand and clarify the requirements early in the development process. Deficiencies & contradictions in the requirements were identified & corrected. This draft documentation was of course updated as required before delivery to the customer along with the code.

[Charlie Clark]

Re: Worst of Both Worlds?

There is indeed something known as "requirements engineering" but I guess it got thrown out with the trash years ago. To be fair, done right "user stories" can help.

[Anonymous Coward]

Re: Worst of Both Worlds?

"... first task allocated to me was writing user documentation for software which hadn't yet been written."

This used to be standard for all 'Newbies' when joining the development team !!!

It hit multiple targets at the same time:

1. You got up to speed with what was being delivered to the Customer, at a reasonable high level.

2. You learnt who knows what within the team as you discoverd more & more information.

3. You learnt 1st hand the difference between the original requirements & what was actually being delivered.

4. You learnt WHY the requirements had to be varied from the original.

5. The documentation was progressed, at least a first pass of it :=) and you demonstrated your value to the team while you were learning.

:)

[heyrick]

Re: Worst of Both Worlds?

"There is no excuse or 'magic' process that makes 'Documentation' no longer needed !!!"

This.

There is very little code around with a suitable level of commenting that no documentation is necessary.

And when you get those people who think that comments aren't necessary because the code is obvious and self documenting? Grrrrr! That's the worst sort of mess to have to delve into, especially if the programmer spent far too much time "doing clever things" so there's the code, the total lack of comments on WTF is going on, and all sorts of possible potential side effects. The last time I came across code like that, I looked at what the API was trying to do and just rewrote all the damn functions. Sometimes life is too short......

[Scene it all]

Re: Worst of Both Worlds?

The comments (embedded, within the code, not in separate documents or header file) need to explain the PURPOSE of the code - not just what it does, WHY it does it this particular way, and under what conditions it can be executed. Those are concepts that can not be expressed in any programming language I know.

Without that information, the code is un-reviewable and un-maintainable, EVEN LATER BY THE PEOPLE WHO WROTE IT.

[Ken Hagan]

Re: Worst of Both Worlds?

There are many problems for which a decent description would require rich text, formulae, tables and diagrams. Please don't try to use ASCII art in a code comment for this. Use the proper tool for the job.

Feel free to add code comments that cross reference the proper document.

Oh look, I've just re-invented Literate Programming. Don't worry though, it'll never catch on.

[Paul Hovnanian]

Re: Worst of Both Worlds?

"un-maintainable, EVEN LATER BY THE PEOPLE WHO WROTE IT."

And so now they want to hire code commenters when, in the past, we couldn't even get the meat-sack devs to comment their own code?

Let us know how that is working out.

[amanfromMars 1]

Just asking ..... for when there be no comments accompanying and explaining the dislike vote

Does El Reg tolerate and approve non-human generated down votes on comments?

[heyrick]

Re: Just asking ..... for when there be no comments accompanying and explaining the dislike vote

You'd have to be pretty sad to create a bot to go downvoting a bunch of comments on a tech website.

Though, maybe this explains "that string of one downvote" on comments critical of a certain drug-using techbro? Yeah, he's about sad enough and has the money to pull it off. Shall I expect to see one lonely downvote just here:

[Will Godfrey]

Re: Just asking ..... for when there be no comments accompanying and explaining the dislike vote

Don't tempt them

[Paul Hovnanian]

Re: Just asking ..... for when there be no comments accompanying and explaining the dislike vote

"You'd have to be pretty sad to create a bot to go downvoting a bunch of comments on a tech website."

Not when there are so many willing to do so for free.

[Charlie Clark]

Documentation, meh. Show me the tests

YMMV but when reviewing software I like to look at the tests to see if the code does what it says it should. If the tests are okay, and assuming they pass, then they can form the basis of documentation.

[Will Godfrey]

Re: Documentation, meh. Show me the tests

Upvoted but with a proviso. You need to be sure that the right things are being tested.

Some years ago I made a bug report giving details of how to duplicate it. Specific controls had to be set in a certain way. To their credit, it was fixed fairly quickly but I got a note back from one of the developers :-

"We didn't think anyone would do that."

[Charlie Clark]

Re: Documentation, meh. Show me the tests

All true, but that's why I like to read the tests.

[Anonymous Coward]

Re: Documentation, meh. Show me the tests

Downvote because 'tests' show that the code does what you thought it should do ... BUT without valid documentation are you able to verify that the code is doing what the original requirements asked for OR if the requirements had to be varied, why they had to be changed, what the change was and that the customer agreed to the change.

Concentrating ONLY on the tests is simply setting yourself up for failure IF the customer says 'Yes' the code works BUT it is not what I asked for !!!

Documentation is important for EVERYONE ... it is the source of all TRUTH ... Today & Tomorrow !!!

Your 'Meh' implies that it does not matter ... this is a mistake !!!

:)

[Charlie Clark]

Re: Documentation, meh. Show me the tests

I didn't say I concentrate only on the documentation just that I prefer to start with the tests when reviewing the code. I've read a lot (and written some) documentation that was inaccurate and misleading; tests are either right or wrong. My main point is about using tests to check that the code does what the developer thought it should.

Requirements is a whole other area, that is all to frequently neglected, as I mentioned in a separate post.

[doublelayer]

Re: Documentation, meh. Show me the tests

What kind of tests are you expecting to see? There are a lot of things that documentation should contain which tests don't say anything about because documentation operates at the user level rather than the internal structures level. Code can implement a search algorithm, properly test the algorithm, properly plumb that algorithm to the interface, and properly test that the interface controls haven't become disconnected from the system that runs the algorithm for the user. If you write docs based on what you test, then you can explain the interface to the user: "The input to the search algorithm goes in the text box labeled Search Query, and pressing the button labeled Search sends that query to the algorithm" I.E. the part you could likely leave out and users would figure it out anyway. Unless the user knows what inputs do what to the algorithm, they are still lost, and a narrative tutorial and a well-organized reference for all options will often be more useful than anything you would build from tests.

[Will Godfrey]

Re: Documentation, meh. Show me the tests

Some documentation is for users. Other documentation is for informing new devs and reminding existing devs why decisions were made as much as what they were.

Also, what testing is available, how to implement it and what the limitations are.

[doublelayer]

Re: Documentation, meh. Show me the tests

Even that should be built from more than just test cases. There's a lot of information about what components are for and how you operate the development and release process. Some of that tests can show you but often don't because many programs have much more unit testing or basic integration testing than test cases that follow massive flows*. Other things simply can't be demonstrated by tests because they're not related to that.

* Most of the time, that focus on testing small things is the most logical approach for two different reasons. One is that, if the tests on each unit are correct and the tests between each set of interacting units are correct, then the behavior of the whole program should be correct by induction. The second is that massive flows are the most likely to change, so if you spend too much of the test budget on those, you'll have to change them for any change in desired operation and they won't be as stable. However, in practice, it also means that the smallest tests are the most likely to be implemented. Unit tests are the most likely to be called out in coverage reports than integration tests and integration tests rarely cover all the possible ways for different units to interact.

[Charlie Clark]

Re: Documentation, meh. Show me the tests

I prefer unit tests, because in my experience you get a better understanding of the software writing and reading them, but I've seen plenty of code where monolithic (and barely comprehensible) tests got sufficient coverage. With a good layer of unit tests, it's much easier to add both integration and user tests to help find out what you may have missed because refactoring and rewriting are inevitable.

[trevorde]

Human generated doc-slop

Worked on a project where everything had to be documented. Typical example was:

/// <summary>

/// Gets or sets the SetPoint

/// </summary>

public int SetPoint ( get; set; }

* what is the range of allowable values?

* what happens if you try and set it to a value outside the allowable range?

* what is the default value?

Turned out the only real requirement was documentation had to be present. It didn't have to be correct, relevant or useful.

[Mister Goldiloxx]

Just because you can do something, it doesn't mean that you should.

The end result is machines who can think, and people who can't...