back to article Microsoft shrinks AI down to pocket size with Phi-3 Mini

Microsoft claims the latest incarnation of its lightweight Phi-3 Mini AI model rivals competitors such as GPT-3.5 while being small enough to be deployed on a phone. Phi-3 Mini is a 3.8 billion-parameter language model trained on 3.3 trillion tokens. This figure is up from the 2.7 billion parameters of Phi-2, which Microsoft …

  1. elsergiovolador Silver badge

    Future

    Imagine the future, where you can sit back, enjoy a nice cold non-alcoholic drink, while you both put your AI devices on the table to do all the arguing for you.

    Meanwhile, you can just peacefully stare at the sunset being sure that whoever is truly right, the AI will sort this out.

    1. Casca Silver badge

      Re: Future

      Perfection. At least if you can mute them

  2. steelpillow Silver badge
    Trollface

    Given the Microsoft idea of reasoning that we all know and love, I doubt this AI needs anything bigger than a troll's tit to run on.

    1. Rich 11 Silver badge

      Just as long as it wasn't built to run in IE6...

  3. Throatwarbler Mangrove Silver badge
    Terminator

    Reasoning

    Nowhere in the article did I see a definition of "reasoning," which I would consider quite important when deploying an AI model. What is it actually (supposed to be) doing?

    1. steelpillow Silver badge

      Re: Reasoning

      What is it actually (supposed to be) doing? This, probably:

      "Hi, My name's Alix. I am your AI assistant. How can I help you today?"

    2. Bebu Silver badge
      Holmes

      Re: Reasoning

      《Nowhere in the article did I see a definition of "reasoning," 》

      I would have hoped that reasoning here would be some form of logic formally defined (or a collection of such logics.) Its not as though logic doesn't leave one spoilt for choice - modal logics, various probabilistic and fuzzy logics, temporal logics etc etc.

      Soundness (no hallucinations) and completeness (all valid conclusions can be derived) are nice to have. ;)

      Off course the whole ChatGPT AI/LLM approach is more of the suck-it-and-see school of philosophy while, like an unparticular courtesan, not too fastidious about what it sucks..

    3. Justthefacts Silver badge

      Re: Reasoning

      Target tasks are whatever a human could do by bashing keys on their keyboard. Obviously, that varies from job to job. I literally googled a list of “22 middle-class jobs worth exploring” and what tasks the one paragraph job spec listed:

      Book publisher - sift through the mounds of dross submissions to find the very few potential gems. Machine Shortlist will be analysed more fully by a human. You know what the current coping mechanism at publishers is? They arbitrarily drop half of submissions in the bin, unread. Because what else if all you have is the time in the day.

      Book editor - read what your authors have written, provide basic copy-editing and advice (like autocorrect, but also fact-check, consistency check, stylistic, readability), or indeed the author can have this on their own desktop - hint: this is already deployed, and first indications are that books edited this way are *significantly* better edited than human editors - both sales and book critic reviews.

      Interpreter - simultaneous interpreting is a *major* blocker both for doctors, and particular our court system. Currently up to 10% of all court trials are abandoned as ineffective because the right set of interpreters are not available for some accused or witness. In a medical context, the alternative to machine translation in practice, isn’t “a better translation by a human”. It’s that an interpreter isn’t available, and the patient doesn’t get treated. Simple as that.

      Office Manager - “plan meetings, make travel arrangements, process payroll, answer telephones, create the work schedule for administrative staff, office procurement, and manage file systems.” See anything that an AI *couldn’t* do? It doesn’t have to be as good as a human office manager, just good enough to avoid paying a whole salary, viz supermarket self-checkout.

      Welcome to the real world.

  4. HuBo
    Unhappy

    small enough to be deployed on a phone

    But not small enough (at 2.3GB) to be downloaded through a phone's 3G/4G connection ... "ollama run phi3" ... Error: max retries exceeded: Get https://...r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/ ... ?

    1. HuBo
      Pirate

      Solved: small enough to be deployed on a phone

      Got it. In case others run through these issues, here's my flow to get Phi3 running in ollama, in CPU-only mode, on a Whiskey Lake i3 (2 SMT cores), connected to the web via a 3G/4G smartphone. It takes about 2.6 GB of downloads, and maybe 5 or 10 GB of disk space (2.8 GB wasn't enough it seems).

      1. download and "install" ollama (290 MB) from its website, into a folder of your choice, following manual instructions (without systemd autostart and such; precede commands with "sudo" if needed/wanted):

      curl -L https://ollama.com/download/ollama-linux-amd64 -o ollama

      chmod +x ollama

      2. download the CPU-only version of Phi3 (2.3 GB) from Hugging Face, the file is named: Phi-3-mini-4k-instruct-q4.gguf (download straight from the website to avoid the "Error: max retries exceeded"), save it in the same folder as ollama.

      3. in that same folder, create a text file named buildPhi3 (for example) that contains the single line:

      FROM Phi-3-mini-4k-instruct-q4.gguf

      4. run ollama to build the Phi3 model (that will go in your home folder, eg. under /home/you/.ollama/models ... you need some free disk space there, or make symlinks to another place):

      ./ollama create Phi3 -f buildPhi3

      5. test that your Phi3 works with a simple prompt: "What is The Register?":

      ./ollama run Phi3 "What is The Register?"

      Voila! Tobias' 10-minute HowTo has more of course. Either way, one should be able to prompt engineer his/her hacking future better from thereon in, or out (at 4x 100% CPU utilization, 5 GB of SDRAM in use!).

      1. druck Silver badge
        Coat

        Re: Solved: small enough to be deployed on a phone

        I was able to skip steps 1 through 5, and then get on with my life quite happily.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like