The artificial intelligence community has a new feather in its cap with the release of Falcon 180B, an open-source large language model (LLM) boasting 180 billion parameters trained on a mountain of data. This powerful newcomer has surpassed prior open-source LLMs on several fronts.

Announced in a blog post by the Hugging Face AI community, Falcon 180B has been released on Hugging Face Hub. The latest-model architecture builds on the previous Falcon series of open source LLMs, leveraging innovations like multiquery attention to scale up to 180 billion parameters trained on 3.5 trillion tokens.

This represents the longest single-epoch pretraining for an open source model to date. To achieve such marks, 4,096 GPUs were used simultaneously for around 7 million GPU hours, using Amazon SageMaker for training and refining.

To put the size of Falcon 180B into perspective, its parameters measure 2.5 times larger than Meta’s LLaMA 2 model. LLaMA 2 was previously considered the most capable open-source LLM after its launch earlier this year, boasting 70 billion parameters trained on 2 trillion tokens.

Falcon 180B surpasses LLaMA 2 and other models in both scale and benchmark performance across a range of natural la

Go to Source to See Full Article
Author: Jose Antonio Lanz

Did you like this?
Tip BTC Newswire with Cryptocurrency

• Bitcoin
• Bitcoin Cash
• Ethereum
• Litecoin
• Monero
• ZCash

Donate Bitcoin to BTC Newswire

Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate via Installed Wallet Copy

[X] Click Here to Hide Donation Details

Donate Bitcoin Cash to BTC Newswire

Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate via Installed Wallet Copy

[X] Click Here to Hide Donation Details

Donate Ethereum to BTC Newswire

Scan the QR code or copy the address below into your wallet to send some Ether:

Donate via Installed Wallet Copy

[X] Click Here to Hide Donation Details

Donate Litecoin to BTC Newswire

Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate via Installed Wallet Copy

[X] Click Here to Hide Donation Details

Donate Monero to BTC Newswire

Scan the QR code or copy the address below into your wallet to send some Monero:

Donate via Installed Wallet Copy

[X] Click Here to Hide Donation Details

Donate ZCash to BTC Newswire

Scan the QR code or copy the address below into your wallet to send some ZCash:

Copy ZCash Address

[X] Click Here to Hide Donation Details