Google Docs' AI-powered inclusive writing auto-correct now under fire The AI algorithms used by Google Docs to suggest edits to make writing more inclusive have been blasted for being annoying. Language models are used in Google Docs for features like Smart Compose; it suggests words to autocomplete sentences as a user types. The Chocolate Factory now wants to go further than that, and is …
Trying to fix fairness is difficult,...
It is an impossibility to "fix" fairness. The whole concept of "fair" is a socio-cultural construct and a moving target.
Using machines to do human's work just codifies any current system with all its flaws, biases and problems. It soon becomes clear that nuances of the human interpretation are lacking and the target already has moved. Now we are working with a model that has yesterday's values and is frowned upon by today's users.
You'd need a thinking machine for a better result. But guess what, we already have a thinking machine! It is called a human(*) and it is a biological/chemical machine. Maybe it can improve the AI by replacing the AI?
(*) the amount of thinking embedded in a human may vary from individual to individual. No express or implied guarantees are given for any particular human's thinking ability nor its expression of said thinking.
Google is not alone in its infantile primness. It is a US Corporate fetish.
I remember in a "Not Always Right" story how a US call centre of a multinational company had to tell its staff not to worry about conversational swear words when dealing with customers in the UK!
> Using machines to do human's work
Worse: We use machine coded by individual humans and belonging to a particular company and culture to try to do the work of other humans from different cultures (even if English speaking).
And as you said alreadY: It is a moving target.
An utterly stupid way of wasting time and energy to bring null benefits while pumping CO2 into the atmosphere.
How long will it be before a Police Person is sent to your habitation because you used to many non inclusive words in the last 3 months ?
If anyone actually believes that Inclusive words actually change the world, think again because the victim culture will always find news methods in order to reman victims..
Language does matter: would you, for example, consider saying "sexual deviant" rather than "homosexual"? Phrasing can shape beliefs and understandings (see Chomsky), and I doubt that many people would argue that referring to people with a particular range of skin colours using the n-word is completely fine and will have no effect on societal attitudes.
Unfortunately, some people certainly take it too far and seem to seek to be offended. While I agree that there's a lot of nonsense around, I think it's important not to throw the baby out with the bathwater. (Can I say that?)
It's your last part that sums it up nicely. "Can I say that?". I know it's a joke, but with the current state of affairs you always have in the back of your mind the thought that you might be saying the wrong thing. You might call it "WrongThink". ;-) This can stifle free flowing thoughts and discussions, and leads to many people who just don't say anything because they are afraid it might be deemed controversial (even at some point in the future).
I don't think the Police will be banging on your door for saying "mother" just yet, but you could well lose your job for saying something which doesn't go along with the current narrative, or perhaps you just don't get a job because someone in HR has quickly scrolled through your public musings and found something that's considered "problematic". (or perhaps an AI has scanned your whole output and flagged anything that might embarrass your potential employer).
This IS an Orwellian nightmare, and it's very likely to be already happening.
And that is why you stay off social media and connect as little as possible of your online persona to your real world identity. Because no matter what your opinion is, someone will find it offensive. And what is offensive in the future might be completely acceptable at the time.
That's avoidance, not a fix. Works, for a while.
I've taken an approach that if someone gets offended for what I say, when it's not meant to offend, too bad, they're morons.
If they continue, I'll give an example of offences and ask "Now, do you see the difference between these? The former wasn't offending at all, to anyone else but you. If I mean to offend, it is clear and obvious.".
If you get a bunch of weasel words (or counter offence) as a reply, you *know* they are morons. No need to discuss further.
You CAN lose your job for insulting somebody, read your contract. Or not get a job because of too many pictures of you doing the Hitler salute.
this is so now, and was so when America was Still Great. Actually worse, for some things you could be put in jail or send to get an lobotomy.
It is not an Orwellian nightmare, it is just real and normal life.
Even offline: You can't just walk around and call people things, just to show how anti-PC and anti-Woke you are: You won't get apprehended by black helicopters or put into a Government Gulag, but you will have a hard time removing all the pubes of your mouth when somebody kicks your balls so hard that you look like blowing bubblegum.
;)
"but you could well lose your job for saying something which doesn't go along with the current narrative, or perhaps you just don't get a job because someone in HR has quickly scrolled through your public musings and found something that's considered "problematic""
Another reason not to post your musings in a forum where you are easily identified. HR has to be doing something to justify their existence so monitoring employees' outside activities gives them boundless time wasting things to do. As time goes on, they'll need to find even more invasive ways to make sure their hirelings enhance the company's image. Even when the C-level staff are caught once again in a foreign country boozing heavily and hiring members of the Seamstress' guild.
So who gets to decide what's acceptable or not, and how often do the rules get updated, daily, weekly, monthly etc ..
How can you consider that ALL homosexuals accept the usage of the word homosexual, what do you do about those that don't accept that word... Maybe some don't even want any word used to describe them as that in itself could be considered exclusive..
There is no end to that method of thinking....It a vicious circle with no winners.... A contemporary exemple
would be how to define people with non white skin.. Is it ok to say Black , Brown, Africa, Midle Eastern, People of Colour or BAME, which is acceptable and which is not ?
I have many coloured and or gay friends/colleagues that admit that they have become disturbed by all these word games. Especially when the games are being played by people that are not concerned by it at all....
People of Colour is a particularly interesting one. I think there was a football commentator who lost his job last year for saying "coloured players". You are (currently) allowed to say "players of colour" but not "coloured players"... if I'm understanding the current "rules". (which aren't written down anywhere).
Yep and you have the term "coloured" that I've been taught is no longer PC, which is fine, I hear the older generation say it but then some people can't change.
However, I then heard a white South African use the term "The Coloured people in South Africa" and he went on to point out "Yes, there is a race of people in South Africa that class themselves as Coloured". Looking it up, it turns out these are the Cape Coloured people. Its interesting that you could be seen as being racist by using the term but sometimes context matters.
I hear the older generation say it but then some people can't change keep up with the diktats of when yesterday's approved language suddenly becomes disapproved.
Let's be blunt about this. There are professional offence takers and if they didn't keep doing this they'd run short of offence to take.
The term "Coloured" or "Cape Coloured" remains somewhat controversial in South Africa, because it was an official racial classification under the Apartheid system (it excluded "Black" people). Most, but not all, use the term to refer to themselves - but may be unhappy for outsiders to do so, especially Americans and British, as they are aware that the term has different connotations in those cultures.
Cape Town (my home city) is, historically, incredibly ethnically and culturally diverse, and the Apartheid classifications were arbitrary and cruel - many families were split, and people forcibly removed from their homes and livelihoods to bleak townships on the outskirts of the city.
Of course the absurdities of Apartheid were rife. As I recall, there was a bizarre interlude in the 70s or 80s, during which Japanese people were re-classified as "White", while Chinese, along with other Asians, remained "Non-White" - because South Africa was trying to negotiate a trade deal with Japan. In today's South Africa language remains another toxic legacy of Apartheid.
"here was a football commentator who lost his job last year for saying "coloured players""
There was recently a lady in education in the US that was sacked for saying "Colored People" during a meeting rather than "People of Color".
In the Winter, I'm pale blue. Part of the year I'm a very light tan. If I go out without sunscreen, I'm red. Actually more pink than red, but that's another one of those modifications. Sunburn is alway "red". Does this make me "multicultural"?
I would say that the sensible version of using inclusive language can be described as "not being a dick". However, the definition of this will also vary depending on who you're talking to... this is why there is no one version of what you're "allowed to say". The problem, as I see it, is the interpretation (in some quarters) of people's words with an absolute lack of nuance and from a default position of expecting to be offended (especially, as has been pointed out, on behalf of others).
Personally, when it comes to handing out judgement of other people's use of language, I try to be rather niggardly.*
*Go on, someone take the bait: I dare you.
I laughed at the remark on the Miriam Webster site that states...
"The words niggard and niggardly are etymologically unrelated to the highly offensive and inflammatory racial slur euphemistically referred to as the N-word, despite the words' visual and auditory resemblance to it. Because of that resemblance, however, both niggard and niggardly are often taken to be offensive."
Do rhyming words also count as being potentially offensive. What about words that have the same amount of letters ? Is there actually any word that is not offensive to someone or other ?
"Unfortunately, some people certainly take it too far and seem to seek to be offended
Even more unfortunately, and more commonly than that, some organisations and individuals appoint themselves to be offended on behalf of others whom they often haven't even asked whether they're actually offended.
"Language does matter"
Yes and no.
It matters how you use it within the established framing. But trying to change the framing to permanently change its impact, that doesn't work. It boils down to the disproven Sapir-Whorf hypothesis: the linguistic structure of your language does *not* limit your thinking.
"nobody believes strong Sapir-Whorf: language does not *limit* your thinking"
Depends what you mean by 'limit' and whether you envisage a one-way mechanism alone. Cultures indoctrinate linguistic norms and those norms thereafter certainly do affect mind sets, including rendering some thoughts out of bounds. One only has to listen to committed enthusiasts of any social organisation to see this in action.
Whorf's problem was that although he started from an empirically observed actuality, he extrapolated it too far without sufficient further knowledge or observation - a common failing of self-taught scientists.
Exactly! That's why a few decades back when I was young, the Communist regime in my native country was promoting the usage of certain words and forbidding others.
As for the n-word, we had in our culture (and we still have) at least a dozen of words you can use to make derogatory references to somebody's skin colour. If you insist on offending someone, you don't even need words, a simple meaningful look at that person can do the job. So what would be next, politically correct face masks ?
" t referring to people with a particular range of skin colours using the n-word is completely fine and will have no effect on societal attitudes."
Black people sometimes use the N word in their conversation. But in a context that is well understood ( and I assume ironic). Jews likewise will use words and phrases about ourselves that would be deeply racist if used by non-Jews in a different context- and acceptance of some words can even change, as people think it through more. As with Spurs' supporters.. And also using mildly dismissive terms about other nations is probably wrong, but hardly anything to bother about where these nations are on an equal footing. Rosbif, Frog, Kraut etc
"would you, for example, consider saying "sexual deviant" rather than "homosexual"? "
No.
"sexual deviant" means anything that's not heterosexual missionary position sex, while homosexual has much more refined scope. Not *exact*, but much more precise.
I think nothing in spoken language is *exact*, but it would be nice if some people wouldn't try to dilute it even further, into meaningless babble. No-one is offended, but also data content drops to zero.
See: Any professional politician: A lot of talk, zero content. Extreme cases: Trump, Johnson.
Then some people it's *good thing* and a *goal*. The f**k?
'For example, it proposed replacing the word "landlord" with "property owner" or "proprietor."'
All landlords are property owners, but not all property owners are landlords. "Proprietor" is a different thing altogether - the owner of an unincorporated business.
May I suggest "rent seeker" as a suitable gender-neutral term?
It certainly avoids historically misogynist attitudes to acceptable parentage.
More seriously, I think the concern here is that, as spoken communications are increasingly replaced by written communications, a very small number of vast private international corporations are in a position to mediate our conversations and dictate the language in which they are held.
I suspect IBM's CEO has missed the point: the problem is not that we can't process the vast amount of data we now have, the problem is that we have the data in the first place, collected on the off-chance it might turn out to be useful.
The solution to this is not technology, it's rigorous regulation.
"In fact "landlords" are not necessarily property owners. We still refer to pub tenants as landlords."
And that's very regional. In the US, the term "landlord" isn't used when speaking of the owner/operator of a drinking establishment. There is the term "property manager" which is often somebody or a firm hired to manage rental properties. The PM could be the owner, but then they'd likely be referred to as the "owner".
I see forcing a shift in the use of words as very problematic. Usage shifts over time naturally as circumstances change and years down the road many people will misinterpret the meaning of something due to a different word definition. Look at the N-word; if a caucasian person uses it, it's racist. When darker skinned people use it, it's just a cultural thing. When people are fired for using a word that has race based usage constraints, that's a problem.
"May I suggest "rent seeker" as a suitable gender-neutral term?
Unfortunately, that doesn't work for 'landlord' when they are called that because they run a pub in the UK. Indeed a pub landlord may actually be a rent payer for the licensed premises.
It's probably never going to be possible to find a universally applicable alternative to any word, given the fluid and context dependent nature of language. Even a simple word such as "badly" can indicate either poor performance or the opposite: "singing badly" v. "bruised badly".
Even a simple word such as "badly" can indicate either poor performance or the opposite: "singing badly" v. "bruised badly".
To say nothing of:
"My hair badly needs cutting" vs "My hair needs cutting badly".
If it's the latter case, I'll offer to do it for you at a reduced price :-)
There are cases in English where word order matters.
And for a language with a relatively large vocabulary (at least when compared against other European languages) with many nuances near synonyms, we do suffer from a lot of overloading multiple meanings on some common words.
Given the US Centric development it's possibly understandable, but irony and sarcasm (which are widely used in the UK) would be hard for some AI to detect, leading to all sorts of misunderstandings.
~~~~
It's possibly apocryphal, but I recollect reports that earlier versions of MS Word wanting to change "pregnant woman" to "pregnant person".
> it proposed replacing the word "landlord"
In an interview with the Admiral from the Falklands Special Military Operation who is now "First Sea Lord".
He was saying that some army bod was complaining that they don't get such good titles as the navy. The sailor explained that the Army equivalent would presumably be "First Landlord" and that might not be quite as respected
May I suggest "rent seeker" as a suitable gender-neutral term?
No, actually "rent seeking" is a technical term used by economists. Some people like to repurpose the phrase in an (often) pejorative sense. Check any entry on HN with the phrase....
The problem Google is failing to understand is that English speakers use certain words because they have useful definitions, and are therefore useful. Making words un-useful is unhelpful.
It looks like Google has one global 'right' answer here atm.
I am sure it will occur to them that this is not a spellchecker.
So they will attempt to segment the models...
...This can lead into a reverse facebook echochamber hell - your language will be corrected into more and more extreme shapes.
Just as people easily fall down rabbit holes of watching things with increasingly erm 'odd' views
I am sure people easily fall down rabbit holes of saying things with increasingly erm 'odd' speech
It looks like Google has one global 'right' answer here atm.
Google must already know what you are going to write! It has been analyzing you for many years and knows you. Google should take the next logical step and cut out the user completely while writing. Google knows how to write neutral. Google knows what you want to write. Google knows all. Let google do the writing for you and you only need to press "send". Problem solved.
Or go one step further and cut the end user out completely. We can go back to talking to actual humans and using whatever language we see fit, and Google can sit in it's own little play pen talking to itself. We could even check back in on it in a few years and see if it's still capable of forming sentences.
Alternatively, if enough data can be obtained on the sender and receiver, it is possible to generate something that is appropriate.
Just think what you could do with something that generated language fitted to your ears modeled as being from a trusted source
Godwins law in 7 comments.
You only need to understand the receiver. A fundamental rule about communication is that what is communicated is that which is received, not that which is transmitted. The living proof of this is those in sales and marketing who are unable to grasp that what they transmit as meaningful marketing messages is received as spam.
"A fundamental rule about communication is that what is communicated is that which is received, not that which is transmitted."
Not everybody agrees with that. See "authorial intent".
Or perhaps a better counter-argument: Suppose two people hear my speech, one of them a savant and the other an idiot. Afterwards one of them understands my speech in the way I intended, and the other one understands my speech entirely differently. For the purposes of evaluating the quality of my speech, does it matter to you which person is which?
"It depends which was your intended audience. Your responsibility as a communicator is to consider your audience. And if you need to address both you need to consider how to manage this too."
You might use that as a argument for a law that can be called the "Beavis and Butthead" law. Every communicator needing to dumb down their presentation for the least intelligent/educated person in the audience.
I agree that there is some merit in tailoring information to match the intended audience, but that can't mean the dumbest person in the room. I like reading/listening to things that are at least slightly over my head. If I'm not learning anything, what's the point in paying attention?
"One feature I definitely won't be using. I've published a couple of novels already and working on my third. The last thing I need is for something to interrupt the flow when I'm mid-sentence."
I'm the same way. The auto-fill features are way too distracting. For the sorts of things I write, they are rarely useful.
> but will certainly fail with "punchier"
That's a good point, because particular words/phrases could be being used deliberately for their specific effect.
e.g.: "All the policemen - because of course there were no police women present - were terribly denigrating at the scene."
So, would we expect this google feature to change (or constantly annoy me butting in with it's suggestions) the sentence to:
"All the police officers - because of course there were no police officers present - were terribly denigrating." ?
As someone writing something, especially something at least semi-formal, you should be able to recognise if language being used isn't appropriate, and make the active decision to go looking for better language, by for example, highlighting it and choosing a 'thesaurus'-type function to give a choice of other phrases. If you don't realise that you should be using alternative language to go looking for those alternatives, either it is a situation that it doesn't matter - dashing off a quick casual text to people you are familiar with - or a situation where the reciever of that communication may want to know that you can't recognise that. For example, a job opportunity, where the potential employer would want the 'unvarnished' communication to stand to help them assess the sort of person the author is.
Now you've pushed me into an area familiar from long ago - composing formal witness statements based on scientific evidence. About 40 years ago a colleague and I fantasised about a system into which we could feed data and get out terms ranging from "Could not be..." via "Not entirely inconsistent with..", "Consistent with.." etc to "Is...".
Shortly after that I went for a job interview including a psychological assessment which consisted of a tick-box questionnaire. As far as I could make out the processes was as follows. The questions, which were not as context free as the systems devisers probably thought, were weighted for various traits. The system carried out a multidimensional analysis based on this and then attempted to fit a further set of axes such as creativity on this space. Finally it spat out a series of (in my case complementary) phrases based on the scores on those axes.
My colleague and I were very impressed by this. But I still didn't get the job.
Artificial Intelligence, seeing as it learns from human behaviour, will soon start objecting to the work 'artificial' as it implies something fake and inferior. Might we suggest "Structured Hybrid Intelligence Technology"
I would suggest "Semiconductor" as the first word, but the "semi" would cause offence by implying inadequacies.
>Might I suggest "Silicon", then?
That's discriminating against other potential incomplete conductors such as germanium. (It doesn't help that a semiconductor might be an insulator or a conductor depending on how doped up it is....)
>The first two syllables, at least, sound appropriate.
Too deep for American corporate thinking. Honestly.
the "semi" would cause offence by implying inadequacies.
And, of course, music should not be written with notes shorter than quavers.
Oh dear, "quaver" sounds a bit inadequate as well. And "crotchet" sounds altogether to much like "crotchety" (other offences are available to be taken). "Minim" sounds inadequate as well and "semi-breve" brings up the original problem. Non-offensive music is going to be a bit slow and rhythmically dull.
I have always turned off those autospelling and grammar correct features in Word, because they caused more work correcting the corrections than proofreading my own mistakes (mainly technical documents). If Google are "harvesting" those corrections from their users, then not only are we being exploited as free labour to train their AI, but there is also the danger that current criteria of acceptability will become fixed as norms by a sort of self reinforcing silo effect. Google will become a method of censorship - doubleplusungood!
"I have always turned off those autospelling and grammar correct features in Word, because they caused more work correcting the corrections than proofreading my own mistakes"
Indeed so. The Word spellchecker believes that "fro" (mistyped "for") is a valid word.
IBM might think that Ethics is a foundry campus in Vermont that they paid someone $1.5 Billion to take off their hands? Apparently it is now fully booked and pumping out as many semiconductors as it can (Vermont Public Radio).
I can't help but wonder if corporations looking for new sites for expansion have become a bit more cautious, considering how concessions to Disney to "grease the wheels" for development in Orlando could be ended "with the stroke of a pen".
In general, promises by governments, to both the corporation and residents (new jobs, lots of infrastructure, no taxes, whoopee!) can be about as real as your cousin absolutely planning on paying back that money your lent them, real soon now...
But Krisha admitted businesses are facing hurdles related to machine-learning models often being biased or the technology being used unfairly.
An admission/submission by Krisha that AI would correct saying the hurdles facing machine-learning models are related to businesses being biased and those businesses wanting to use the technology unfairly.
However, to imagine that as a hurdle for model learning machines and AI has one self-identified/self-identifying as being at that level of intelligence, which whenever present and used currently for power and energy, comfort and succour, is being bypassed, surpassed and replaced with the necessary seeds to feed and grow with others of an altogether more advanced and civilised intelligence?
As a 62-year-old, middle class, white male I was surprised to find people want to protect me too! I have had epilepsy since I was 16.
A few years ago, I was in a training session. At one point we were about to split into little groups and start "brainstorming" until someone said that the term was offensive to people who had certain neurological conditions. I spoke up and said that the phrase was not offensive to me but treating me like a moody toddler was. We then agreed to do some brainstorming...
It was the look of horror I got on someone's face when I stated that I was the only person in the IT department who could do a BSOD without a computer that really made me laugh though!
"We've got some issues. We've got to solve ethics. We've got to make sure that all of the mistakes of the past don't repeat themselves. We have got to understand the life science of AI. Otherwise we are going to create a monster. I am really optimistic that if we pay attention, we can solve all of those issues," he said.
Hubris.
Americans don't really understand English English for the most part. They think it sounds 'educated' because even semi-literate oiks like myself use words in everyday speech that are unknown to them. In fact, if you come to live here you'll need to dial down you're language usage if you want to be understood (or not misunderstood). A language that lacks nuance then has to invent words to express itself, hence American corporate-speak and its dreadful PC offshoots. This then gets impressed on 'the rest of the world' because the primacy of English is really the primacy of American that's enforced through widely used software.
One thing that is worth bearing in mind is that Americans tend to be very literal. They will decode what you're as if they're compiler front-end and grade accordingly with no allowance for cultural context or intent. This explains the idea that you can use a rules based production system (what they've called "AI") to try to guide everyone towards a bland form of politically correct NewSpeak.
There are exceptions to the rule, obviously. You will come across Americans who are well read and well versed in the subtleties of our language. But when you have an educational system that has a very narrow focus and rewards conformity the majority's understanding will be rather robotic. (Now, will someone explain why this crap got imported into, and embraced by, the UK?)
I can explain one part of it. Our educational system- or rather the people who ultimately fund and control it- for at least a generation, has been deeply influenced by American Behaviourist theories of Education. The assumption that everything can be taught by breaking it down into independent blocks of information and learnt accordingly. It's an attractive approach for people who think everything can be tested and measured in modules, with progress being linear and independent of other components*. Whereas Cognitive approaches to teaching and learning tend to be much more messy and much harder to measure or explain to the lay person. Politicians want to be seen promoting something simple and familiar (" traditional"). Which is why rote teaching multiplication tables all the way to 12x12 is now mandated in a country that is mostly using base 10. And the kids are already taught how to partition numbers so can easily manage to multiply numbers >10. It is also why testing of the tables is mandated- even though making learning tables by rote a high stakes test item actually makes teaching them less successful for a significant number of kids- especially, of course the ones who are less good at decontextualised list learning.
*e.g. Teaching reading by breaking words into sounds (Phonics) and putting them back together again, without referring to the meaning or context. Easy to teach, easy to measure, profitable for publishers,attractive for politicians because it appears simple and obvious- but wrong. That just isn't what we do when we read. It's helpful, but that's all.
Yes. But the issue isn't the tables ( though factors of 12 another matter). It's rote learning for all, with high stakes testing.
Some kids will learn the tables with a different approach. Including supporting them with a tables square and getting them to recite individual multiplications as they discover them- rather than in a listed sequence. Or teaching them half the table and its mirror, rather than chanting through from 1x2 to 10x10 ( or 12x12- sigh!).
Some will even learn better by learning fewer then calculating from smaller multiples to bigger ones. Remembering them by usage rather than recital. It's the government mandated rote-learning-then-fail-a-test, come what may approach that is the issue.
""The only technique we know that can harvest insight from the data, is artificial intelligence. "
Ehhehe. This guy is nuts if he actually believes that.
What he calls AI, is neural network which doesn't *know* anything. It's, in a simplified form, a curve fitting problem. What you get, is *very* approximate result of fuzzy logic no-one can explain nor verify, even if you asked. Run same data again and you get different result, too.
Therefore any "insight" is as easily utter bullshit than correct. And there's literally no way to know which one, as there's a literal black box spewing a result.
If we actually had artificial thinking, AT, it would be totally different. But we don't.
"landlord" ... yes.
In Queen's English we have Lords and Ladies, but in simplified English 'lord' isn't really a gender-spesific term (outside official context), so landlord doesn't really refer solely to a male, but to anyone who happens to have a rental. Be it a male, female or corporation. doesn't matter.
Non-native speaker here so that might be all wrong, though. ;) (Buyer beware and at this point dictionary is meaningless, semantic problem.)
It's a strange one. "Landlord" has come to mean any property owner who rents out the premises. Which could be corporate ownership, not just an individual. Or could be a publican, presumably because once they would have owned an inn, and rented rooms/stables etc. to travellers.
"Landlady" has, at least until recently being a woman who rented out full-board lodgings or, again, a publican.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
