back to article Website programming? Pffft, so 2011. Python's main squeeze is now data science, apparently

The Python programming language is now used more for data science than web development, according to a new survey. The poll, conducted by the Python Foundation and JetBrains (which makes the PyCharm IDE among other things) involved over twenty thousand Python developers from more than 150 different countries. It asked …

  1. Tim99 Silver badge

    “the surging popularity of Flask, a bare-bones web framework ”

    It might also be be worthwhile to have a look at Bottle: bottlepy.org.

    1. Charlie Clark Silver badge

      Re: “the surging popularity of Flask, a bare-bones web framework ”

      Nah, they both use globals to make life easier to throw something together. Pyramid is the best framework if you prepared to write a couple of lines of configuration. It's also famousy not made by aliens.

      1. Tim99 Silver badge

        Re: “the surging popularity of Flask, a bare-bones web framework ”

        "It's also famousy not made by aliens." ???

        1. eldakka

          Re: “the surging popularity of Flask, a bare-bones web framework ”

          I'm guessing the OP meant:

          The Egyptian Pyramids are claimed by some (crackpotty) people to have been made by aliens.

          There is a framework called Pyramid, and this framework does not claim to have been made by aliens, even tho it is also a Pyramid. The only thing it shares with the Pyramids of Egypt is the name, Pyramid, and not the mythology.

          But, unless the OP chimes in, I guess we'll never know ;)

      2. Someone Else Silver badge

        Re: “the surging popularity of Flask, a bare-bones web framework ”

        It's also famousy [sic] not made by aliens.

        What? We have Martians writing Python code?!? What will they think of next?

        All this time, I thought AMfM was a C++ guru...

        1. eldakka
          Alien

          Re: “the surging popularity of Flask, a bare-bones web framework ”

          > What? We have Martians writing Python code?!? What will they think of next?

          That would certainly explain the martian errors I was seeing a a linux /var/log/messages recently.

    2. jimbo60

      Re: “the surging popularity of Flask, a bare-bones web framework ”

      Or, if you just want a lightweight RESTful API and not a GUI, look at Falcon Web Framework, falconframework.org. A full API can be done in one small file; no templates, no models, etc..

  2. werdsmith Silver badge

    Our Uni is preparing to abandon MatLab to move to Python and libraries.

    I am now trying to use Python as much as possible, I don't actually like it that much because I have so many years invested in C and C++ that I can usually do things much easier they are like a comfortable old jacket and it's the easy route for me to use them. I need to get to the same point with Python to keep up.

    1. Czrly

      Do it. Never look back!

      Seriously, just leave Matlab in the ditch on the side of the road.

      My wife (currently completing her PhD in pattern recognition and deep learning) has half her code in MatLab and at least half the time-cost of working on that portion can be attributed to hassles with licensing.

      The other half is in Python.

      She can just work on the Python half, wherever she is, even without a connection. She can easily punt it to the powerful GPU-packing machines in her lab (or, when they're not free, my gaming PC with its 1080ti) to train models overnight. She can even do this remotely by simply issuing a `git pull` from Jupyter's terminal emulator in a web browser -- that's all it takes and the notebook that acts as a pretty `main()` method is ready to run. Heck, she can navigate to the little Gentoo server that lives on the top of a cupboard in our house, patch bugs in GitLab's browser-based editor, commit from the web interface and restart an experiment all from her phone while sitting on a train!

      It's almost like she owns her own work, in Python.

      And, if she copies some Python snippets off the web (StackOverflow, reddit, GitHub or Kaggle, for example), the license doesn't belong to MathWorks. Read the terms of MatLab's community forums; all code posted becomes property of MathWorks automatically! (That's proper arrogance and arseholery.)

      MatLab does only one thing well: it visualises data in a way that no open-source tool does. When you're stopped on a breakpoint or looking at variables in your session, drilling down into them, graphing them, pretty-printing them and watching them in MatLab makes for a better experience than any other.

      If open-source tools like Jupyter were to implement the fancy data visualisation tricks that MatLab provides, I'm sure that MathWorks would be reduced to changing their corporate font, refreshing their logo, updating their branding and patent-trolling to stay alive -- and good riddance!

      1. werdsmith Silver badge

        Re: Do it. Never look back!

        I personally actually swerved Matlab and use Octave. And Wolfram a bit too.

        But yes, I'm coming round too Python now after years of resistance. Mainly because of its broad adoption.

      2. Cuddles

        Re: Do it. Never look back!

        "She can just work on the Python half, wherever she is, even without a connection..."

        You can do all that very easily with Matlab as well.

        "And, if she copies some Python snippets off the web (StackOverflow, reddit, GitHub or Kaggle, for example), the license doesn't belong to MathWorks. Read the terms of MatLab's community forums; all code posted becomes property of MathWorks automatically!"

        Can you provide evidence for this? According to the actual Matlab Central terms and conditions, anything you post falls under Creative Commons Attribution Share Alike 3.0 license, except for the answers section which falls under BSD. Far from trying to claim ownership of your code, there are several parts in the terms explicitly denying any and all responsibility for anything. Last updated over 2 years ago, so this isn't a new thing.

        https://uk.mathworks.com/matlabcentral/termsofuse.html

        We're also moving away from Matlab, mainly to Python, but that's simply down to the one big problem Matlab has - cost. That used to be justified since there were a lot of features in Matlab that just weren't easily available in any non-proprietary platform. But there are now Python libraries to replicated pretty much any feature you might want, so paying hundreds of thousands per year for software licenses just doesn't make sense any more. The price keeps increasing, but the gap between their feature set and free Python features is smaller than ever before.

      3. Sometimes an Engineer

        Re: Do it. Never look back!

        "MatLab does only one thing well: it visualises data in a way that no open-source tool does."

        It took me a while to like python (and there are still things I dislike) and it is only numpy/scipy/pandas that really shine for me, but now it's my go to for quick data analysis. However if there is one thing I'd sell my grandma for, it would be for python to be as good as Matlab at visualizing data. That for me is where Matlab shines (that and simulink).

        Of course I could just sell my grandma to afford a Matlab license, but then I'd have to use Matlab.

    2. Anonymous Coward
      Anonymous Coward

      "I have so many years invested in C and C++".

      Then use that! I'm not sure how Python or any scripting language is going to perform better than those. When it comes down to pure math, the lower the better. What happens to Python if everyone all at once wants to go lower? If all you want is the general sting manipulation over the math code, I'd question Python to start, as it is one of the worst with strings.

      I use python for many, many utilities, but I just don't see how using it in a computational environment is desired, at least not anymore than any other scripting language.

      1. ibmalone

        If all you want is the general sting manipulation over the math code, I'd question Python to start, as it is one of the worst with strings.

        It's kind of funny to see that a couple of responses down from people discussing Matlab... I can use Matlab, but its handling of strings (and really, anything other than numbers, but strings are a particular pain), is the thing that prevented me ever becoming fond of it.

        Libraries, python has two strengths, it does do compilation (but is still a virtual machine with all the abstractions as overhead), and many of the libraries are written in C, so you can get good performance from libraries. I still prefer R for analysis, particularly since tidyverse packages make passing datasets around and specifying models very easy (takes a while to learn its idioms though, and it seems full of special behaviours), but can see why python might gradually overtake it as it's now pretty much the teaching language for lots of non-compsci programming courses, so the availability and coverage of libraries is going to keep improving.

    3. jimbo60

      Re: I have so many years invested in C and C++

      Same as I, 30+ years of C/C++. But if you are fluent in using OO-style C++, it's about half a day of learning to be reasonably proficient at OO-style Python, which is how it should be used. And SO much easier to use for string and symbolic manipulation because the language takes care of all the memory allocation and garbage collection.

  3. HmmmYes

    I like Pyhton.

    I say that, as over the yars, Ive had to elarn Java and Javascript.

    And support 100k+ of C++ of varioius vintage.

    1. TRT Silver badge

      Good job Python isn't a syntax Nazi.

      1. Tom 7

        Re Good job Python isn't a syntax Nazi.

        No its an indentation Nazi which is just as bad. It just shifts the problem somewhere else, it doesnt get rid of it, Because its not really a problem, Its the underlying concepts that people find difficult.

        I have been teaching some youngsters and most prefer brackets when things get a bit complicated, it may be my influence I'll grant you, but when a code block stretched to a few tens of lines parentheses matching is a god send. Who needs a collapsible editor when you can find a matching delimiter. And why do collapsible editors only ever collapse from the top?

        1. Anonymous Coward
          Alien

          Re: Re Good job Python isn't a syntax Nazi.

          I used to think this. Then I saw code written by scientists in languages which don't enforce indentation.

          1. TRT Silver badge

            Re: Re Good job Python isn't a syntax Nazi.

            Yeah, but repeated whitespace introduces hidden redundancy, which HTML thankfully eliminates. And I kind of think that should be the same for code really. Non-displaying characters should only play a minimum role in the construction of a language. But that's just my opinion, really. At the end of the day it's about readability.

          2. theModge

            Re: Re Good job Python isn't a syntax Nazi.

            Wait till you see code written by hardware engineers *shudders*

            1. Anonymous Coward
              Anonymous Coward

              Re: Re Good job Python isn't a syntax Nazi.

              Wait till you see code written by "professional" software "developers"

              Literally random indentation and spacing in the Java code in a very large product I work on. It's not even consistent within statements.

              And you are discouraged from using the IDE reformat functionality as it breaks the code review process, since they use a crappy diff algorithm.

              Anon because Larry would beat my ass.

              1. Tim99 Silver badge

                Re: Re Good job Python isn't a syntax Nazi.

                "Literally random indentation and spacing in the Java code in a very large product I work on. It's not even consistent within statements."

                Cut and paste from a multitude of sources?

          3. Someone Else Silver badge

            @tfb -- Re: Re Good job Python isn't a syntax Nazi.

            Which just goes to show...you can write FORTRAN in any language.

            And don't for a second think Python is suddenly exempt from that axiom just because it "enforces" indentation.

            1. Anonymous Coward
              Anonymous Coward

              Re: @tfb -- Re Good job Python isn't a syntax Nazi.

              Of course it's not. But it does mean that the Fortran that gets written will, for instance, not have the first non-white character of every line in column 6 (OK, labels, I know).

          4. W.S.Gosset

            Re: Re Good job Python isn't a syntax Nazi.

            > I used to think this. Then I saw code written by scientists in languages which don't enforce indentation.

            Awww HELL yeah. I actually wrote a monster sed script* once to fix/do the indenting on 10yrs of inherited scripts which all basically just fell down the lefthandside of the page. Utterly unmaintainable.

            * at which point I discovered sed is actually an entire language, with loops and so on.

            1. jimbo60

              Re: Re Good job Python isn't a syntax Nazi.

              Hee hee. I once used a modified sed (modified to add proper regular expressions, back in the day when it definitely did not have that) script to convert Bliss to C. Those were the good old days :).

        2. Tom 38

          Re: Re Good job Python isn't a syntax Nazi.

          When things get too complicated that indentation is a problem for you, refactor so they are no longer so complicated.

          1. doublelayer Silver badge

            Re: Re Good job Python isn't a syntax Nazi.

            I agree, but sometimes you don't have that authority. I recently had to work with code where the writer (I don't know what is wrong with them, but something clearly is) had mixed tabs and spaces with *every* number of indenting spaces from 1 to 17 and several lines indented at random numbers above 17. This was C code, and the indentation didn't match up all that well with the brackets. Thankfully, this code was put under my branch, so I was able to thoroughly reindent the thing. However, I wouldn't mind something external preventing them from doing that to the code in the first place.

          2. Gene Cash Silver badge

            Re: Re Good job Python isn't a syntax Nazi.

            > When things get too complicated

            It's nice when we can work on small and simple things that we have complete control over everything, including code style, isn't it?

            Some day you'll work on a real software project where other people just don't give a shit except pumping out LOC and management does not enforce its own style guidelines, and when you reformat things, you get beat up for too many code changes. But they pay well, so I just have another bottle.

            1. Tom 38

              Re: Re Good job Python isn't a syntax Nazi.

              15 years of working on real software projects, thanks very much. Maybe someday you'll get to work somewhere where you can influence code quality standards too.

        3. Anonymous Coward
          Anonymous Coward

          Re: Re Good job Python isn't a syntax Nazi.

          "No its an indentation Nazi which is just as bad."

          At work a couple of years ago someone did a couple of "introductio to python" lunch and learn sessions. I remember coming out of the first thinking "its Lisp without the brackets"

    2. karlkarl Silver badge

      Vintage C++ is *much* easier to maintain than vintage Python.

      Try running even a simple python 1.x script in the latest 3.x interpreter.

      Many isues.. and thats the easy bit.

      The big issue with vintage python is the obsolete dependencies it drags in. Everything is a sodding external package and everything sodding no longer exists.

      Basically the only way to maintain vintage python is a rewrite. This isnt *always* the case with C++.

  4. Paul Johnston

    Find this bit hard to believe

    It says " it is number one in the statistical domain,"

    I'm surprised R doesn't fulfill this role.

    1. TRT Silver badge

      Re: Find this bit hard to believe

      It's a bit more advanced than just an adder.

      1. Alister

        Re: Find this bit hard to believe

        Yeah, Pythons tend to be bigger than Adders...

    2. Tom 7

      Re: Find this bit hard to believe

      It does but programming is no longer about pathetic arguments about whose syntax or who squares the circle better, its about access to libraries and python is mostly OK for that - though reinventing DLL hell is not a good move.

    3. owlz

      Re: Find this bit hard to believe

      Algorithm 1: Empirical tool selection for data science

      ========================================

      Input: n hours of your own time, a bunch of free tools

      1. Use pandas + matplotlib + scikit-learn to solve some motivating problem for an period of n/2 hours

      2. Use dplyr + ggplot2 + mlr to approach the same problem for the same duration

      3. Evaluate and compare results of (1) and (2)

      4. Agree to never tell anyone else (on the internet or in real life) what tool they should be using.

  5. Bronek Kozicki
    Windows

    If you like Python, learn Go

    It has two benefits:

    1) no silly indentation rules

    2) static type safety

    Now, Go for data science ... I am afraid it currently does not have nearly as many useful libraries as Python does, and you are unlikely to be able to use it in a Jupyter notebook. So, perhaps stick with Python (but learn Go anyway).

    1. Charlie Clark Silver badge

      Re: If you like Python, learn Go

      Go is becoming very popular in some domains. particularly engineering systems where the static typing allows for faster binaries, but is unlikely ever to gain as much acceptance as Python for precisely the reasons you give.

      When it comes to the ML stuff, all the heavy lifting is done by specialised code running on the metal so no advantage to be gained by switching the scripting language,

    2. Kevin McMurtrie Silver badge

      Re: If you like Python, learn Go

      My experience with Golang is that its crude data structures and lack of exception handling makes it a nightmare for complex tasks. Python is very easy for complex tasks, as long as the codebase doesn't get beyond the practical size limits for dynamically typed languages.

      1. Anonymous Coward
        Terminator

        Re: If you like Python, learn Go

        What are thesr 'practical size limits'? I've used machines whose operating systems were written in dynamically-typed languages.

  6. steviebuk Silver badge

    Confused

    "The Python programming language is now used more for data science than web development, according to a new survey."

    I thought it was always mainly for data science. But then I haven't touched programming since the 90s and that was while learning Pascal :) so what do I know.

    1. Version 1.0 Silver badge

      Re: Confused

      Everyone I know is using Python in biomechanics data analysis - a lot of old code has been rewriten ... I must admit I'd forgotten that it was once used for web sites.

    2. OnlyMee

      Re: Confused

      Here are some larger traffic sites (actually bad word) most what is build with Django or Flask count as applications/application backends rather than websites.

      https://www.netguru.com/blog/top-10-django-apps-and-why-companies-are-betting-on-this-framework

      https://www.quickstart.com/blog/10-famous-websites-built-with-python/

  7. 89724102172714182892114I7551670349743096734346773478647892349863592355648544996312855148587659264921

    Yeah, keep Python off the net. Damn thing is a sieve.

    1. Omgwtfbbqtime

      More of a colander I thought.

  8. Anonymous Coward
    Anonymous Coward

    Here on Wall Street...

    The fairly standard arrangement amongst prop traders and hedge funds is that there's a bunch of people like me providing latency-critical infrastructure in C++, binding the exposed parts to Python for the benefit of researchers and traders. The latter two are required to be decent programmers but are primarily mathematicians and strategists, so giving them something without the extra baggage of build systems and 20 years of strictly backwards-compatible syntax, and with the breadth of statistical functionality introduced by modules such as Pandas, is a big win. Especially when it comes with a REPL and various REPLs-that-graph are available (Jupyter being a fairly visible one). The relative ease with which the two languages can be bridged helps a lot in mediating between the two divergent skill sets at this sort of firm.

    That's not exclusively the arrangement — e.g. Jane Street has an enviable dedication to functional programming via OCaml — but it's probably used by 90% of low-latency trading firms.

    1. Anonymous Coward
      Anonymous Coward

      Re: Here on Wall Street...

      Python is a glue language. Write the core stuff in C++ and allow others to operate it from Python. Keeps it simple. Does however mean you ideally need C++ programmers though.

  9. Anonymous Coward
    Alien

    Python now used more for data science

    Not only that, Python is also used on the communication devices on Counterpart.

    1. AHuman

      Re: Python now used more for data science

      Hmmm... The [communication devices](https://i.ibb.co/LkY1s6n/Counterpart.jpg) image you linked to is actually showing Perl code, not Python. That said, I'll have to check out that TV series.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like