Posts by fg_swe

AI query optimization in IBM's Db2 shows you can teach a tech dinosaur new tricks

Application ?

Elaborate.

Other Approaches

Safely Transform Query Plan by

+Monte Carlo

+Genetic Mutation

+Simulated Annealing

Re: AI optimizing SQL err well

You can still run EXPLAIN to see the generated plan, I guess.

The AI will not be fully freewheeling...

Re: DB2 has included "DB2" since 1983!

1.) If the AI is only allowed to perform "safe" transformations of the query plan, there is limited worry regarding AI "correctness". No sane software engineer would grant an AI "full authority query plan transformation". AIs are "mostly correct", not "perfect".

2.) ANNs might indeed be a new type of heuristic for query plan optimization. Much more "rule of thumb based" than "statistically, logically based". It will prolly work nicely on standard scenarios and will prolly fail badly on special, extreme cases.

Elaborate

According to your terminology, an optimizing compiler also includes "AI" ?

1.) My understanding is that the query plan mangling will never change end result, but only runtime/resource consumption. AI can only be used for heuristics, similar to existing heuristics in traditional optimizers.

2.) I have seen large enterprises where they run convoluted queries and still wait for hours in queue to get results. The business questions are similar to "how many millions did we lose in winter, in the US, because a component was not that great ?". There exist corporations who really need large scale relational databases to better understand their core business. Much more than just a funny CMS.

Re: Query Plan Optimization Details

Now, if these flimsy web 2.0/key-value/graph "databases" contain 1/1000th of DB/2 query optimizer technology I would be surprised.

Again, relational databases are modern day wonders. They continue to be improved by scientists and engineers, as they are still far from perfect. Most users, including software engineers, never really cared about this technology. They just assume "it works" and only complain when the optimizer fails on an especially convoluted SQL query.

SQL database are on par with the latest jet engines, 3nm semiconductors, metal 3D printing, gas chromatography !

"Obvious"

To your wet computer of 100E9 Neurons and 100E13 Synapses there exists an "obvious" query plan. But for the dumb CPU there might be Billions of alternatives and each of them must be "rated"/"estimated" and then the best one chosen. In reality, the dumb CPU will use some sort of heuristic to boil the billions down to 10000 different plans. Works often, but not always, well.

The premise is that the AI is essentially a novel heuristic for find a good query plan. Any experience from the real world ?

Query Plan Optimization Details

https://www.tridex.org/wp-content/uploads/Db2-Query-Optimization-101.pdf

Well

SQL Database Servers are "as old" as Unix kernels such as Linux, MacOS X and later, iOS.

I am quite sure there are serious teams at IBM, Oracle, MSFT developing incremental improvements of their SQL engines. I would not be surprised to learn that serious percentages of code are not older than five years.

Regarding "Dinosaur"

I fully disagree with this notion. Advanced SQL Database are some of the most advanced systems in existence. As complex as a modern operating system. Critical for almost ANY business operation from accounting to inventory to sales statistics to analytical processing etc.

Relational databases continue to lead the field of databases due to solid theory, good performance, good query plan optimizers and so on.

The "new" key-value databases are merely useful for building massive collections of cat picture sharing systems for teenagers.

Any True DB/2 Experts Around ?

So far I can only read generic nonsense in this forum.

What I can say:

1.) Query Plan Optimization is a Hard Problem. MySQL fails badly in this aspect for complex queries.

2.) For many queries it would be too runtime-expensive to evaluate all possible solutions; heuristics are used.

3.) Maybe AI can add "novel heuristics".

4.) Any query plan, runtime-efficient or not, will produce the same result. AI is not messing with results.

5.) Adding further indices to a DB schema will quickly be a double-edged thing, as index maintenance will also consume serious runtime.

Microsoft answered Congress' questions on security. Now the White House needs to act

Repost

"The most expensive thing that can happen to a government agency is to lose sensitive data to a competing government.

For details, you can ask Karl Dönitz and Isoroku Yamamoto.

Windows must be banned from processing any secret government information, as they are at least 20 years behind the state of the art."

Corruption

Open Source does not have the bribery infrastructure in place. Unlike the $corporation.

Also, good people avoid the government, as it cannot pay competitive wages. Instead they hire armies of losers.

When push comes to shove, a college-dropout oligarch will be the Effective Surgeon General, as we have seen with COVID.

A dark world full of corruption.

Or Is It ?

Maybe they know all the backdoors and how to effectively plug them ?

Outside the Pentagon+NSA, everybody else is a victim to be pwned.

Or(more likely) they are simply corrupt and MSFT pays best.

Microsoft bigwig says the Feds catching Chinese spies in Exchange Online is the cloud working as intended

Oligarchy Disinformation Operations And Malgovernance

Oligarch bribing newspapers: https://www.berliner-zeitung.de/news/gates-stiftung-unterstuetzt-den-spiegel-mit-weiteren-29-millionen-dollar-li.194183

Oligarch buying goverment:

https://www.infosperber.ch/wirtschaft/konzerne/who-geraet-immer-mehr-in-abhaengigkeit-von-bill-gates-co/

https://www.welt.de/politik/deutschland/plus209247817/Umstrittene-Finanzierung-Das-Gates-Dilemma-der-WHO.html

Of course this is absolutely, never, ever related to his Pharma Investments !

Secure Government Email

Postfix

GNUpg

DeltaChat (also uses GNUog)

But alas, no kickbacks possible !

CISA - Censorship America

CISA coordinated with Facebook, Google, youtube, Apple and others in order to squelch any anti-Covid messages of ordinary NATO citizens.

https://judiciary.house.gov/media/press-releases/new-report-reveals-cisa-tried-cover-censorship-practices

https://eu.usatoday.com/story/money/2023/10/03/fifth-circuit-cisa-ruling-biden-first-amendment/71051110007/

https://www.theregister.com/2023/10/04/cisa_barred_from_coordinating_with/

It was essentially a CIA-NSA-ARMY operation designed to facilitate illegal censorship. They seconded their operatives into CISA, so they could avoid legal trouble.

This backfired big time, as people simply used TELEGRAM, provided by Mr Durov "out of Dubai" (believe this at your own cost).

Yes, that's true. Free speech provided by Russia because CIA censors the h3ll out of American services.

US Space Force wanted $77M to reinforce GPS – and Congress shot it down

Alternative Navigation Means

That GNSS is rather brittle has been known for a long time. Not really expensive or complex to jam.

It's time to develop other means. Often this is just a software upgrade of an existing sensor processing chain.

Not Completely Correct

You can do "fancy" things with waveforms, you can use redundancy in the coding, you can use secret waveform creation(depending on a key) and you can use directional antennas.

All of which helps massively against jammers.

But in the ideal case for the jammer (being very close to the receiver), this does not work. That's true.

A tale of two missions: Starliner and Starship both achieve milestones

Too Reasonable

Your post contains too much reason. Next you will refer to the dozens of faiiled A4/V2 launches !

Von Braun was such a loser, I tell you !

Databricks' $1B Tabular buy raises questions around table format wars

Re: Can Somebody Translate From Gibberish

I realize I am behind the bullshit cycle.

It's actually a useful concept: https://www.coursera.org/articles/data-lake-vs-data-warehouse?utm_medium=sem&utm_source=gg&utm_campaign=B2C_EMEA__coursera_FTCOF_career-academy_pmax-multiple-audiences-country-multi&campaignid=20858198824&adgroupid=&device=c&keyword=&matchtype=&network=x&devicemodel=&adposition=&creativeid=&hide_mobile_promo&gad_source=1&gclid=EAIaIQobChMIjb-ii-TEhgMVCbRoCR0qyBAxEAAYBCAAEgLQCvD_BwE

Can Somebody Translate From Gibberish

"Data lake" ?

The Canon Cat – remembering the computer that tried to banish mice

F.U.D. ?

1.) Apple is one of the highest capitalized companies on the globe. They rake in fantastic profits. Something they must do right.

2.) Apple hardware and software looks nice and is ergonomic, unlike most competitors. I know from personal use of Linux, Windows, Apple.

3.) I did a bit of MFC and a bit of the GNU Next UI clone. The Next thing was much better.

4.) I also used Motif, which was horrible compared to Next.

5.) Of course where is light, there are shadows. The walled garden of iOS, for example.

6.) Android is a knock-off of iOS. Jobs/Apple invented it. All your explanations cannot change that.

iphone

Jobs also made the Smartphone happen, while Nokia was asleep at the wheel.

He could imagine a software-heavy phone, he knew there should be a single appstore, a touch UI etc. An always on internet connection.

All based on Next, to the present day.

One of the greatest men of the western world, because he proved all the collectivists who inhabit the bank industry wrong. The MBAs ran NOKIA into the ground.

False /Next

Next is the root of both MacOS 10 and later AND iOS. Arguably two of the most important products of our age, measured by user count, revenue and profits.

Also, Next is a leap forward in technology as compared to the Unix contraptions of its time, but it is still a powerful Unix at the core. Much better than X11/Motif and similar GUI monstrosities.

Jobs was a genius and Canon could see that. Not sure they lost their investment.

Compare Next to the drudgery of HPUX/AIX/SOLARIS and the beancounters who led those enterprises. Then you can see the light of the jobs genius.

All of that is true even if you consider him to "just" be the orchestrator of Next.

Codd almighty! Has it been half a century of SQL already?

SQL As Turing Complete Language

Apparently it is a factor of 100 slower than LISP for general purpose computing tasks like Prime Number calculation. SQL was never intended as a general purpose language, so that's OK for me.

In the imagined "SQL/LISP" environment, SQL would be used for "data reading and writing from permanent storage" while LISP would be used for "complex computation".

There would be at least three software engineers who could make good use of this approach ;-)

Query Cache

"Modern" (1995 or later) RDBMS will cache the query and (importantly) the query plan derived from the query.

That reduces query reading+preparation to e.g. murmurhash on the query string. Or just crc32 built into cpu.

"Turing complete"

Similar to C++ templates, m4 or brainfuck ?

PL/SQL: Imperative Language

Isn't PL/SQL an imperative/procedural language running inside the Oracle query executor?

It would be surely interesting to have LISP/SQL in the same setting, though. Lisp is functional.

See this page for language classification:

http://sappeur.di-fg.de/classification.html

Stifling Beijing in cyberspace is now British intelligence’s number-one mission

Dear GCHQ

First clean up the SSL/TLS abomination. It is designed to be very hard to implement securely. For a long time it was effectively an open front door. There exist much better alternative concepts from Europe.

Then proceed to smoke out C in the kernels, go for microkernels. Have a look at Oberon, its ingeniously compact.

But do you really have the will, the minds and the money ?

AWS to pump billions into sovereign cloud for Germany

Klaut

The latest U.S. nonsense, too complex to secure. See recentt Office365 hack.

Great sales scheme, though.

Microsoft's Brad Smith summoned by Homeland Security committee over 'cascade' of infosec failures

Secure Instant Messenger

I suggest using DeltaChat for that purpose. It employs GNUpg for end to end encryption.

Probably 100x more secure than the TEAMS contraption.

Dump CloudCr4ap

The tools for secure comunications are already part of Linux:

+ end to end GNUpg encryption. Never trust a $hitty email server.

+ AppArmor Sandboxing of all apps from Firefox to LibertyOffice to Thunderbird.

+ Dump Microsoft, as they are easily 20 years behind the state of the art. Sandboxing for starters. This is the only language they understand.

+ Never trust the cloud for anything sensitive or more. Rather, run ssh/scp based file servers INSIDE your network.

+ Monitor all traffic at the firewall, maybe AI can detect unusual patterns there.

Alibaba Yitian 710 rated fastest Arm server CPU in the cloud (for now)

"small increase in clock speed"

First and foremost, a CPU must run reliably. Only if that is true, you can fiddle with runtime.

IBM and LzLabs to clash in UK court over Software Defined Mainframe

Please Explain

How does this work on the OS and database level?

Do the Swiss also emulate MVS, DB/2, CICS and so on ?

Or just the S/390 CPU ?

Microsoft teases deepfake AI that's too powerful to release

The FIX

Cryptographic Signing of all Multimedia Material. Similar to digital software signing.

Existing tooling+organizations such as CAs can be used for this purpose.

Microsoft is a national security threat, says ex-White House cyber policy director

Steve Jobs

He was running a tight ship while he was alive. He surely knew how to assemble great technologies and technologists. A great attention to detail, while capable of seeing the big picture*. A giant businessman in the best sense.

All while being a total a$$ relative to his daughter, before you accuse me of worshipping.

*seeing the smartphone before Nokia could see it.

You

...are way too reasonable. Next thing you will require individually locked office doors in government agencies !

"Expensive"

The most expensive thing that can happen to a government agency is to lose sensitive data to a competing government.

For details, you can ask Karl Dönitz and Isoroku Yamamoto.

Windows must be banned from processing any secret government information, as they are at least 20 years behind the state of the art.

Really Hard ?

I am using Windows at work and Linux and Apple at home. If you are open-minded, it is not hard to switch away from Windows.

The fearmongering is MSFT S.O.P. to keep the sheeple inside of their semi-golden(below 0,01mm gold it is made out of steel) cage.

FALSE

So in your mind it is "marketing driven" to be careful about UI changes ? I guess you never heard of ergonomics.

Apple understands that computers must be secure and usable. Also, they understand people want nicely looking design. Not these ugly black Wintel contraptions.

Indeed

VBA is more than good enough to encrypt your files, alter your files or send them directly to the attacker.

Also, it can connect to any ODBC database and mess with your databases, too.

VBA security is more than 20 years behind the state of the art(sandboxing).

Linux Support

Let me repeat this: there are probably tens of thousands of highly experienced Linux software engineers in the U.S. alone. Some of them work for the big Linux companies such as Redhat, Oracle, HPE. Many more work for smaller consultancies and many are self-employed.

A lot of them have all the clearances the government needs. Many more are eligible. Your fear-spreading about this aspect is baseless.

The government are idiots when they give more than 30% of their business to a single supplier. Instead, they should have serious Apple, Google Chrome and Linux seat populations. The data center should be a healthy mix of LInux, BSDs, commercial Unix, Windows Server and mainframes.

I have seen the internals of Deutsche Börse, which can easily compete with most government agencies in terms of computing power. Their Linux based trading system works very nicely. So does Google Chrome, Google Search, Facebook and many other large scale cloud systems.

Sure

F.U.D.

Standard Dollarsoft tactic.

The truth is, Microsoft products should be banned from processing secret government information. As detailed, MSFT is 20 years behind the state of the art in information security.

Showing Your Hand

You are simply a paid MSFT propagandist. I can see that from all the talking points you bring up.

Let me shatter just one argument: IBM/Redhat surely has a sufficient number of employees with government/DOD clearance. The cleared ones can reach out to many top class kernel engineers inside Redhat to fix any issue.

Deutsche Börse

Afaik, the entire system runs on Linux by now. Before they used Solaris and VMS. They are leading in their space from a technology and business perspective.

POSIX

A "clean" posix-based program from the 90s will nicely compile and run on any Linux or BSD in 2024, too.

Valgrind runs in 2024 might expose serious memory errors in that program, though.

FALSE / Linux, Open Source Support Contractors

https://ubuntu.com/support

https://www.suse.com/solutions/business-critical-linux/

https://www.credativ.de/open-source-support-center/

https://www.postgresql.org/support/professional_support/europe/

There are plenty of commercial support companies around. Thousands of highly skilled Linux consultants in addition to that.

No, there is a different reason for the lack of love of open source software. It does not look as polished as Windows and Office. Looks are completely deceiving though, almost inversely proportional to actual security.

Like

"Make Calcutta great again" ?