Flash Attention Wajibiya TPU kan ani ka kalan kɛ cogo gɛlɛn na

Flash Attention wajibiyali TPU kan ani ka sira gɛlɛn kalan

optimisation ɲinini ye siren dɔnkili ye ɛntɛrinɛti injiniyɛriw fɛ. A tɛ layidu ta ko tɔnɔ dɔ bɛna fara a kan dɔrɔn, nka a bɛ nisɔndiya sɔrɔ ka fɛnɲɛnamafagalanw kɔrɔta ka kɛɲɛ ni i sago ye. N ka kɔsa in na odisi min kɛra ka Flash Attention waleyali kura dɔ wajibiya — min dabɔra NVIDIA GPUw kama — Google TPU kan, o bangera o fɛn yɛrɛ de la. Laɲini tun ye bonya ye : ka inference pipeline critique dɔ teliya. Nka, o taama in kɛra masterclass ye tiɲɛ gɛlɛnw na modular system design. O ye maana ye min b’a jira mun na Mewayz i n’a fɔ sigida minnu bɛ fɛɛrɛko caman minɛ ani k’u ɲɛnabɔ, olu nafa ka bon jago baara sabatili la.

Siren dɔnkili min bɛ kɛ ɲɛnajɛba la

Flash Attention ye fɛn caman tigɛli ye min bɛ Transformer modɛliw teliya kosɛbɛ ni hakilijagabɔ sɔrɔcogo ɲuman ye. A dabɔra GPU minnu kama, o ye maji saniyalen ye. An ka baarakɛminɛn jɔnjɔn, n’o ye sɛbɛnw dilanni masin ye, o bɛ a jigi da o misaliw kan kosɛbɛ. Ni an ye jatebɔ jatew ye, a kɛra i n’a fɔ bɛnkan in nɔgɔyara : Flash Attention + an ka TPU quota = baara teliya ani musakaw dɔgɔyali. N y’a Dòn, n dalen b’a la ko ni fɛnɲɛnɛma-ko-kɔnɔ-fɛnw bɛ Se ka Kɛ — n’i bɛ kɛlɛ Kɛ ni kernel layouts (kɛrɛnkɛrɛnnenya la) ye, ni memory spaces (hakilila-yɔrɔw) ye, ani XLA compiler (XLA compiler) — n bɛ Se ka nin peg square in kɛ ka Dòn dingɛ bulama dɔ kɔnɔ, min bɛ i n’a fɔ tensor. a daminɛ na, sinsin kɛra fɛɛrɛko seko dɔrɔn de kan, a ma kɛ sistɛmu dusukun tantanni kuntaalajan kan.

Gɛlɛya yebaliw ka kasɛti

" ɲɛtaa " fɔlɔ kɛra dɔlɔminna ye . Dɔgɔkun caman tɛmɛnen kɔfɛ, n ye modɛli dɔ sɔrɔ ka boli. Nka se sɔrɔli kɛra fɛn ye min tɛ foyi ye. Hack tun bɛ se ka tiɲɛ, a tun bɛ kari ni gafemarayɔrɔ kura misɛnnin bɛɛ ye. Min ka jugu ni o bɛɛ ye, o ye drag yebali da pibiliki bɛɛ kan. TPU kode sira min kɛra ka kɛɲɛ ni mɔgɔw sago ye, o kɛra silo ye, o y’an wajibiya ka deployment scripts danfaralenw mara, ka hooks kɔlɔsili, ani hali data-loading logic. Fɛn min tun dabɔra ka kɛ modulu optimisé ye, o kɛra kɛsu nɛrɛma brittle ye. An ye dɛsɛw sɔrɔ minnu bɛ dimi:

• Jahannama debugging : Profiling baarakɛminɛn jɔnjɔnw tun ye fiyentɔw ye an ka ladamu kernel kan , o kɛra sababu ye ka baarakɛcogo kɔsegin kɛ sugo ye ka sɛgɛsɛgɛli kɛ .
• Ekipu buteli : ne dɔrɔn de ye labirinti kode faamuya , ka yiriwali jɔ ni ne tun tɛ sɔrɔ .
• Jɛkabaara juru : sanfɛla ɲɛtaa minnu kɛra modɛli kunba la , olu tun tɛ se ka wuli nɔgɔya la an ka frankenstein TPU foroko la .
• Musow ka musakaw : hakilijagabɔ gundo dɔ bɔra TPU kan , min bangera an ka hakilijagabɔ ɲɛnabɔli la min tɛ ladamu ye , o kɛra sababu ye siɲɛ kelen musaka tɛmɛnen 40% sanni an k' a minɛ .

hakilila modulu : jɛ-ka-baara fanga-dafalen kan

Kalansen jɔnjɔn tun tɛ TPUw walima jateminɛ algorisimuw kan. A tun bɛ modulari de kan. An tun ye sariyakolo jɔnjɔn dɔ tiɲɛ: sistɛmu dɔ kɔnɔfɛnw ka kan ka kɛ fɛn ye min bɛ se ka ɲɔgɔn falen-falen ani ka baara kɛ ɲɔgɔn fɛ, u man kan ka kɛ ɲɔgɔn fɛ. Ni an ye yɔrɔ dɔ wajibiya min tɛ dugukolo yɛrɛ ta ye ka don an ka kulu kɔnɔ, an ye sabatili, jɛya ani teliya saraka walasa ka baara kɛcogo ɲumanba dɔ kɛ min tun man teli ka kɛ sɛnɛko la. O yɔrɔ de la, jago OS modulari i n’a fɔ Mewayz ka filozofi bɛ kɛ ko kɔrɔba ye. Mewayz tɛ ka i datugu kulu kelen kɔnɔ; o ye ka orchestration layer di min b’a to i bɛ se ka baara kɛ ni baarakɛminɛn ɲuman ye baara in na — a kɛra GPU-specific optimization ye wo, a kɛra TPU-native model ye wo — k’a sɔrɔ i yɛrɛ ma connective tissue jɔ ani k’a ladon.

ye

"Fɛɛrɛbɔ min bɛ dɔ fara sigida gɛlɛya kan, a ka c'a la, o ye fɛɛrɛko juru nataw dɔrɔn de ye min bɛ a yɛrɛ kɛ ɲɛtaa ye. Nafa lakika bɛ bɔ ɲɔgɔndan saniyalenw na ani yɔrɔw la minnu bɛ se ka wuli ka bɔ u nɔ na, a tɛ bɔ cɛsiriw la minnu bɛ kɛ siɲɛ kelen."

kalan ni pivot ka taa teliya sabatili la

An labanna ka Flash Attention kɛlɛli jagoyalen in bila shelf la. O nɔ na, ​​an y’an ɲɛsin TPU-native attention implementation ma, hali n’a y’a sɔrɔ a ka surun papiye kan teori siratigɛ la, a y’a jira ko a bɛ se ka da a kan kosɛbɛ, wa a bɛ se ka ladon. Sitimɛ bɛɛ ka baarakɛcogo ɲɛnabɔra tiɲɛ na k’a sababu kɛ a sabatili ye. Min nafa ka bon kosɛbɛ, an y’a daminɛ k’an ka AI baarakɛminɛnw jɔ i n’a fɔ modulu danfaralenw, minnu ɲɛfɔlen don koɲuman. Nin miirili caman tigɛli in — ka bɛnkansɛbɛn saniyalenw bila jɔyɔrɔ fɔlɔ la yɔrɔw ni ɲɔgɔn cɛ ka tɛmɛ baarakɛcogo raw kan, minnu bɛ kɛ sigida la — o de bɛ kɛ tigitigi min b’a to jagokɛlaw bɛ se ka sɛgɛsɛgɛli kɛ ni hakilitigiya ye. Diɲɛ kɔnɔ, min kɔnɔ, fɛnɲɛnɛma-minɛnw bɛ ka wuli joona, i n’a fɔ Mewayz, o bɛ karamɔgɔya di walasa ka seko kura don a kɔnɔ k’a sɔrɔ a ma wotoro jɔ kokura, walima anw ta fan fɛ, k’a sɔrɔ i ma a ɲini ka baarakɛminɛn in da kokura. Sira gɛlɛn y’an kalan ko teliya sabatili tɛ se sɔrɔli ye mikro-kɛlɛ bɛɛ la, nka k’a lajɛ ko i ka kɛlɛbolo bɛɛ bɛ se ka taama ɲɔgɔn fɛ.

Ɲininkali minnu bɛ kɛ tuma caman na

Flash Attention wajibiyali TPU kan ani ka sira gɛlɛn kalan

optimisation ɲinini ye siren dɔnkili ye ɛntɛrinɛti injiniyɛriw fɛ. A tɛ layidu ta ko tɔnɔ dɔ bɛna fara a kan dɔrɔn, nka a bɛ nisɔndiya sɔrɔ ka fɛnɲɛnamafagalanw kɔrɔta ka kɛɲɛ ni i sago ye. N ka kɔsa in na odisi min kɛra ka Flash Attention waleyali kura dɔ wajibiya — min dabɔra NVIDIA GPUw kama — Google TPU kan, o bangera o fɛn yɛrɛ de la. Laɲini tun ye bonya ye : ka inference pipeline critique dɔ teliya. Nka, o taama in kɛra masterclass ye tiɲɛ gɛlɛnw na modular system design. O ye maana ye min b’a jira mun na Mewayz i n’a fɔ sigida minnu bɛ fɛɛrɛko caman minɛ ani k’u ɲɛnabɔ, olu nafa ka bon jago baara sabatili la.

Siren dɔnkili min bɛ kɛ ɲɛnajɛba la

Flash Attention ye fɛn caman tigɛli ye min bɛ Transformer modɛliw teliya kosɛbɛ ni hakilijagabɔ sɔrɔcogo ɲuman ye. A dabɔra GPU minnu kama, o ye maji saniyalen ye. An ka baarakɛminɛn jɔnjɔn, n’o ye sɛbɛnw dilanni masin ye, o bɛ a jigi da o misaliw kan kosɛbɛ. Ni an ye jatebɔ jatew ye, a kɛra i n’a fɔ bɛnkan in nɔgɔyara : Flash Attention + an ka TPU quota = baara teliya ani musakaw dɔgɔyali. N y’a Dòn, n dalen b’a la ko ni fɛnɲɛnɛma-ko-kɔnɔ-fɛnw bɛ Se ka Kɛ — n’i bɛ kɛlɛ Kɛ ni kernel layouts (kɛrɛnkɛrɛnnenya la) ye, ni memory spaces (hakilila-yɔrɔw) ye, ani XLA compiler (XLA compiler) — n bɛ Se ka nin peg square in kɛ ka Dòn dingɛ bulama dɔ kɔnɔ, min bɛ i n’a fɔ tensor. a daminɛ na, sinsin kɛra fɛɛrɛko seko dɔrɔn de kan, a ma kɛ sistɛmu dusukun tantanni kuntaalajan kan.

gɛlɛya yebaliw ka kasɛti

" ɲɛtaa " fɔlɔ kɛra dɔlɔminna ye . Dɔgɔkun caman tɛmɛnen kɔfɛ, n ye modɛli dɔ sɔrɔ ka boli. Nka se sɔrɔli kɛra fɛn ye min tɛ foyi ye. Hack tun bɛ se ka tiɲɛ, a tun bɛ kari ni gafemarayɔrɔ kura misɛnnin bɛɛ ye. Min ka jugu ni o bɛɛ ye, o ye drag yebali da pibiliki bɛɛ kan. TPU kode sira min kɛra ka kɛɲɛ ni mɔgɔw sago ye, o kɛra silo ye, o y’an wajibiya ka deployment scripts danfaralenw mara, ka hooks kɔlɔsili, ani hali data-loading logic. Fɛn min tun dabɔra ka kɛ modulu optimisé ye, o kɛra kɛsu nɛrɛma brittle ye. An ye dɛsɛw sɔrɔ minnu bɛ dimi:

hakilila modulu : jɛ-ka-baara fanga-dafalen kan

Kalansen jɔnjɔn tun tɛ TPUw walima jateminɛ algorisimuw kan. A tun bɛ modulari de kan. An tun ye sariyakolo jɔnjɔn dɔ tiɲɛ: sistɛmu dɔ kɔnɔfɛnw ka kan ka kɛ fɛn ye min bɛ se ka ɲɔgɔn falen-falen ani ka baara kɛ ɲɔgɔn fɛ, u man kan ka kɛ ɲɔgɔn fɛ. Ni an ye yɔrɔ dɔ wajibiya min tɛ dugukolo yɛrɛ ta ye ka don an ka kulu kɔnɔ, an ye sabatili, jɛya ani teliya saraka walasa ka baara kɛcogo ɲumanba dɔ kɛ min tun man teli ka kɛ sɛnɛko la. O yɔrɔ de la, jago OS modulari i n’a fɔ Mewayz ka filozofi bɛ kɛ ko kɔrɔba ye. Mewayz tɛ ka i datugu kulu kelen kɔnɔ; o ye ka orchestration layer di min b’a to i bɛ se ka baara kɛ ni baarakɛminɛn ɲuman ye baara in na — a kɛra GPU-specific optimization ye wo, a kɛra TPU-native model ye wo — k’a sɔrɔ i yɛrɛ ma connective tissue jɔ ani k’a ladon.

kalan ni pivote ka taa teliya sabatili la

An labanna ka Flash Attention kɛlɛli jagoyalen in bila shelf la. O nɔ na, ​​an y’an ɲɛsin TPU-native attention implementation ma, hali n’a y’a sɔrɔ a ka surun papiye kan teori siratigɛ la, a y’a jira ko a bɛ se ka da a kan kosɛbɛ, wa a bɛ se ka ladon. Sitimɛ bɛɛ ka baarakɛcogo ɲɛnabɔra tiɲɛ na k’a sababu kɛ a sabatili ye. Min nafa ka bon kosɛbɛ, an y’a daminɛ k’an ka AI baarakɛminɛnw jɔ i n’a fɔ modulu danfaralenw, minnu ɲɛfɔlen don koɲuman. Nin miirili caman tigɛli in — ka bɛnkansɛbɛn saniyalenw bila jɔyɔrɔ fɔlɔ la yɔrɔw ni ɲɔgɔn cɛ ka tɛmɛ baarakɛcogo raw kan, minnu bɛ kɛ sigida la — o de bɛ kɛ tigitigi min b’a to jagokɛlaw bɛ se ka sɛgɛsɛgɛli kɛ ni hakilitigiya ye. Diɲɛ kɔnɔ, min kɔnɔ, fɛnɲɛnɛma-minɛnw bɛ ka wuli joona, i n’a fɔ Mewayz, o bɛ karamɔgɔya di walasa ka seko kura don a kɔnɔ k’a sɔrɔ a ma wotoro jɔ kokura, walima anw ta fan fɛ, k’a sɔrɔ i ma a ɲini ka baarakɛminɛn in da kokura. Sira gɛlɛn y’an kalan ko teliya sabatili tɛ se sɔrɔli ye mikro-kɛlɛ bɛɛ la, nka k’a lajɛ ko i ka kɛlɛbolo bɛɛ bɛ se ka taama ɲɔgɔn fɛ.