Hugging Face has introduced HUGS, a service similar to Nvidia’s Inference Microservices. HUGS allows users to easily deploy and run large language models on various hardware systems without complicated setups. These containerized model images can be used with popular frameworks like Text Generation Inference and Transformers, making it versatile for different hardware, including Nvidia and AMD GPUs. Although built on open-source technology, HUGS comes at a cost, about $1 per hour per container on cloud platforms. This could be more economical than Nvidia’s services, especially for large models. Hugging Face supports popular models such as Meta’s Llama 3.1 and Google’s Gemma 2. Expect more models to be added soon.
Hugging Face has recently launched HUGS, a new service designed to compete with Nvidia’s Inference Microservices (NIMs). HUGS aims to make it easier for users to deploy and run large language models (LLMs) on various hardware systems. Instead of dealing with complex setups involving tools like vLLM or TensorRT, users can simply use preconfigured container images through Docker or Kubernetes and connect using standard OpenAI API calls.
This new service leverages Hugging Face’s open-source frameworks, namely Text Generation Inference and Transformers, allowing deployment on different hardware setups, including Nvidia and AMD GPUs. There’s also potential support for specialized AI accelerators, such as Amazon’s Inferentia and Google’s TPUs, although support for Intel Gaudi is not currently available.
While HUGS is based on open-source technology, it isn’t free. Users can expect to pay about $1 per hour for each container when deployed on AWS or Google Cloud. This cost is competitive compared to Nvidia’s pricing, which charges $1 per hour per GPU on the cloud or a hefty $4,500 yearly per GPU for on-premises solutions. This pricing structure could make HUGS an attractive option, especially for larger models that require extensive resources.
Hugging Face has also partnered with DigitalOcean to offer its services at a smaller scale, although you will still need to pay for the computing resources. For users who subscribe to Hugging Face’s Enterprise Hub at $20 a month, deploying HUGS on personal infrastructure is an option.
HUGS supports several popular models, such as Meta’s Llama 3.1 and Mistral AI’s Mixtral series. There are future plans to expand support to additional models.
In summary, while you’ll be paying for the convenience of using optimized containers with HUGS, the service promises to make deploying and managing LLMs much more manageable for users across different platforms.
Tags: Hugging Face, HUGS, Nvidia, Inference Microservices, AI deployment, LLMs, container technology, DigitalOcean, open source models.
What is the main focus of the article “Hugging Face puts squeeze on Nvidia’s AI microservice play”?
The article discusses how Hugging Face is competing with Nvidia in the realm of AI microservices, offering tools and platforms that developers can use to build AI applications more easily.
Why is Hugging Face important in the AI space?
Hugging Face is known for its powerful machine learning models and user-friendly libraries, making it easier for developers to access and implement AI technology without needing deep technical skills.
How does Hugging Face differ from Nvidia?
Hugging Face focuses on providing open-source tools and collaborative platforms for AI development, while Nvidia primarily offers hardware and software solutions that optimize AI processing on its graphics cards.
What impact does Hugging Face have on Nvidia’s business?
As more developers turn to Hugging Face for its accessible AI tools, Nvidia may face challenges in maintaining its Market share in the AI software space, as Hugging Face’s offerings become more popular.
Can developers use Hugging Face alongside Nvidia’s products?
Yes, developers can use Hugging Face libraries and models in conjunction with Nvidia hardware to enhance their AI projects, combining the strengths of both platforms for better performance and efficiency.
