Search
Duplicate
๐ŸŽ™๏ธ

AI LipSync-Recognition

ํ•œ ์ค„ ์†Œ๊ฐœ
E2E ์Œ์„ฑ ์ธ์‹์—”์ง„์„ ์ด์šฉํ•œ ์‹ค์‹œ๊ฐ„ ๋ฆฝ์‹ฑํฌ API ์„œ๋น„์Šค ์ž…๋‹ˆ๋‹ค.
์ง„ํ–‰ ๊ธฐ๊ฐ„
2022/03/02 โ†’ 2022/12/02
์œ ํ˜•
UNIV Project
Team Project
ํƒœ๊ทธ
python
pygame
AI

github Link

LipSync-Recognition
PostPlayer

ย E2E ์Œ์„ฑ์ธ์‹ ์—”์ง„์„ ํ™œ์šฉํ•œ Real Time LipSync

ย ๋งก์€ ์—ญํ• 

โ€ข
ํ”„๋ก ํŠธ์—”๋“œ (using Python, Pygame)
โ€ข
์†Œ์ผ“ ํ†ต์‹  (using Python)

ย Engine ์›๋ฆฌ

ย Engine ๊ธฐ๋Šฅ

TTI (Text To Image)

โ€ข
๋ฌธ์ž ์ž…๋ ฅ์„ ๋ฐ›์•„์„œ ๋ฆฝ์‹ฑํฌ ์ด๋ฏธ์ง€๋ฅผ ์ถœ๋ ฅํ•œ๋‹ค

STI (Speech To Image)

โ€ข
๋งํ•˜๋Š” ์Œ์„ฑ์„ ์‹ค์‹œ๊ฐ„์œผ๋กœ ๋ฆฝ์‹ฑํฌ ์ด๋ฏธ์ง€๋ฅผ ์ถœ๋ ฅํ•œ๋‹ค

Two-way & Realtime Communication

โ€ข
์‹ค์‹œ๊ฐ„์œผ๋กœ ์–‘๋ฐฉํ–ฅ์œผ๋กœ ์†Œํ†ต์ด ๊ฐ€๋Šฅํ•˜๋‹ค
โ€ข
์„œ๋กœ์˜ ์บ๋ฆญํ„ฐ์˜ ์ž…๋ชจ์–‘์ด ์›€์ง์ด๋Š” ๊ฒƒ์ด ๋ณด์ธ๋‹ค

STT (Speech To Text)

โ€ข
์Œ์„ฑ ์ž…๋ ฅ์„ ๋ฐ›์•„์„œ ํ…์ŠคํŠธ๋กœ ์ถœ๋ ฅํ•œ๋‹ค

RNN-T Korean

โ€ข
ํ•œ๊ตญ์–ด๋กœ ๋ฒˆ์—ญํ•ด์ฃผ๋Š” ๊ธฐ๋Šฅ

ย ํ•™์Šต ๋ชจ๋ธ

ย ์„œ๋ฒ„ ์‚ฌ์–‘

โ€ข
GPU : 8x NVIDIA A100
โ€ข
RAM : ๊ณ„์ • ๋‹น 5GB
โ€ข
RunTime : ๋ฌดํ•œ
โ€ข
Costs : Free

ย ๊ฒฐ๊ณผ

ย ์ง„ํ–‰ ๊ณผ์ •

ํŒ€๊ด€๋ฆฌ

๊ต์ˆ˜๋‹˜ ํ”ผ๋“œ๋ฐฑ ์ •๋ฆฌ (2)
ํ”„๋กœ์ ํŠธ ๊ฐœ๋ฐœ๊ธฐ (2)
๋กœ๋“œ๋งต (2)
๊ณ„์ • (2)

๊ธฐํšํŒ€

๊ธฐํšํŒ€ ํšŒ์˜๋ก (2)
๋””์ž์ธ ์—์…‹ (2)

๊ฐœ๋ฐœํŒ€

๊ฐœ๋ฐœํŒ€ ํšŒ์˜๋ก (2)
๊ฐœ๋ฐœํŒ€ ์ˆ™์ง€์‚ฌํ•ญ (2)
์ด์Šˆ (2)
์ธ๊ณต์ง€๋Šฅ (2)
ํŒŒ์ด์ฌ (2)
Frontend (2)
Backend (2)
DevOps (2)

๋ ˆํผ๋Ÿฐ์Šค ๋น ๋ฅธ ์ถ”๊ฐ€

espnet github (1)
RNN-T ํ•œ๊ตญ์–ด Transducer (1)
LipSync demo (1)
CTC Algorithms ref (1)
Real Time LipSync (1)
CTC ๊ฐœ๋… ์†Œ๊ฐœ (1)
E2E ์Œ์„ฑ์ธ์‹ ๊ธฐ์ˆ  ์†Œ๊ฐœ (1)
์Œ์„ฑ ์ธ์‹ ๊ธฐ์ดˆ ๊ฐœ๋… ๋ชจ์Œ (1)
AI Hub ํ•œ๊ตญ์–ด ์Œ์„ฑ ๋ฐ์ดํ„ฐ (1)
์ˆœํ™˜์‹ ๊ฒฝ๋ง (1)
beam search ์•Œ๊ณ ๋ฆฌ์ฆ˜ (1)

DB

๋ž˜ํผ๋Ÿฐ์Šค DB (2)
ํšŒ์˜๋ก DB (2)
Docs DB (2)
E2E ์Œ์„ฑ์ธ์‹ ์—”์ง„์„ ํ™œ์šฉํ•œ Real Time LipSync