[MLOps ํ”Œ๋žซํผ Observability & Data Pipeline - 7๋‹จ๊ณ„ : Data Pipeline ๊ตฌ์ถ•]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ S3 Raw ๋ฐ์ดํ„ฐ๋ฅผ Airflow DAG๋กœ ์ถ”์ถœ/๊ฒ€์ฆ/๊ฐ€๊ณต/์ €์žฅํ•˜๋Š” ์—”๋“œํˆฌ์—”๋“œ ๋ฐ์ดํ„ฐ ํŒŒ์ดํ”„๋ผ์ธ์„ ๊ตฌ์ถ•ํ•˜์—ฌ ML ํ•™์Šต ์ž๋™ํ™”์˜ ๋ฐ์ดํ„ฐ ๋ ˆ์ด์–ด๋ฅผ ํ™•๋ณดํ•œ ๊ณผ์ • ์„ ์ˆ˜์ง€์‹ Observability 6๋‹จ๊ณ„: FastAPI Dashboard & Alert Library ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ ๋ชจ๋ธ์˜ ์„ฑ๋Šฅ์€ ๊ฒฐ๊ตญ ๋ฐ์ดํ„ฐ ํ’ˆ์งˆ์—์„œ ๊ฒฐ์ •๋œ๋‹ค. ๊ทธ๋Ÿฐ๋ฐ ๋ฐ์ดํ„ฐ๋ฅผ ์–ด๋–ป๊ฒŒ ์ถ”์ถœํ•˜๊ณ , ๊ฒ€์ฆํ•˜๊ณ , ๊ฐ€๊ณตํ•˜๊ณ , ์ €์žฅํ•˜๋Š”์ง€๊นŒ์ง€ ์ž๋™ํ™”ํ•˜์ง€ ์•Š์œผ๋ฉด MLOps๋Š” ์™„์„ฑ๋˜์ง€ ์•Š๋Š”๋‹ค. ์ด๋ฒˆ ๋‹จ๊ณ„์—์„œ๋Š” Raw S3์—์„œ ์ „์ฒ˜๋ฆฌ, Feature ์ƒ์„ฑ, ์ €์žฅ๊นŒ์ง€ ์ด์–ด์ง€๋Š” ์—”๋“œํˆฌ์—”๋“œ ๋ฐ์ดํ„ฐ ํŒŒ์ดํ”„๋ผ์ธ์„ ์ง์ ‘ ๊ตฌ์ถ•ํ•ด ์ดํ›„ MLflow ํ•™์Šต ์ž๋™ํ™”์™€ ๋ฐ”๋กœ ์—ฐ๊ฒฐ๋  ๋ฐ์ดํ„ฐ ๋ ˆ์ด์–ด๋ฅผ ๋‹ค์กŒ๋‹ค. ...

November 10, 2025 ยท 3 min

[MLOps ์šด์˜ ๊ณ ๋„ํ™” - 5๋‹จ๊ณ„: Airflow ์•ˆ์ •ํ™” & FastAPI HTTPS ๋ณด์•ˆ]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ Airflow Sensor๋ฅผ reschedule ๋ชจ๋“œ๋กœ ๋ฆฌํŒฉํ† ๋งํ•˜๊ณ  ์„ฑ๊ณต ์ฒด์ธ์„ ๋ณด์žฅํ•˜๋Š” DAG ๊ตฌ์กฐ๋ฅผ ๋งŒ๋“ค๊ณ , FastAPI์— cert-manager ๊ธฐ๋ฐ˜ TLS + IP ํ™”์ดํŠธ๋ฆฌ์ŠคํŠธ๋ฅผ ์ ์šฉํ•˜๋Š” ๊ณผ์ •์„ ๋‹ค๋ฃน๋‹ˆ๋‹ค. ์„ ์ˆ˜์ง€์‹ MLOps ์šด์˜ ๊ณ ๋„ํ™” 7๋‹จ๊ณ„: ArgoCD Notifications ์ž๋™ํ™” โ€” ๋ฐฐํฌ ์ƒํƒœ Slack ์ž๋™ ์•Œ๋ฆผ ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ ๋ชจ๋ธ์„ ๋“ฑ๋กํ•˜๋Š” ๊ฒƒ๋ณด๋‹ค ๋” ์ค‘์š”ํ•œ ๊ฑด ์–ธ์ œ Reload๊ฐ€ ์‹คํ–‰๋˜๋А๋ƒ์ด๋‹ค. ๋“ฑ๋ก ์‹คํŒจ๋‚˜ READY ๋ฏธ๋„๋‹ฌ ์ƒํƒœ์—์„œ /reload๊ฐ€ ์‹คํ–‰๋˜๋ฉด ์ž˜๋ชป๋œ ๋ชจ๋ธ์ด ์„œ๋น™๋  ์ˆ˜ ์žˆ๋‹ค. Airflow ํ๋ฆ„์„ ํ•œ ๋ฒˆ ๋” ๋‹จ๋‹จํžˆ ์กฐ์ด๊ณ , FastAPI ์ชฝ์€ HTTPSยทํ™”์ดํŠธ๋ฆฌ์ŠคํŠธ๊นŒ์ง€ ์ ์šฉํ•ด ์•ˆ์ „ํ•˜๊ฒŒ ์ž๋™ํ™”๋œ๋‹ค๋Š” ํ™•์‹ ์„ ์ฃผ๋Š” ๊ตฌ์กฐ๋ฅผ ๋งŒ๋“ ๋‹ค. ...

August 18, 2025 ยท 4 min

[MLOps ์šด์˜ ๊ณ ๋„ํ™” - 3๋‹จ๊ณ„: ๋ชจ๋ธ ๋กค๋ฐฑ ์ž๋™ํ™” (๋“ฑ๋ก ์‹คํŒจ ๋Œ€๋น„ ๋ณต๊ตฌ)]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ ๋ชจ๋ธ ๋“ฑ๋ก ์‹คํŒจ ์‹œ ์ด์ „ ์•ˆ์ • ๋ฒ„์ „์œผ๋กœ ์ž๋™ ๋กค๋ฐฑํ•˜๊ณ  FastAPI์— ์ฆ‰์‹œ ๋ฐ˜์˜ํ•˜๋Š” ๋ณต๊ตฌ ๋ฃจํ”„์™€, ์šด์˜์ž๊ฐ€ ๋ช…์‹œ์ ์œผ๋กœ ๋ฒ„์ „์„ ์ง€์ •ํ•˜๋Š” ์ˆ˜๋™ ๋กค๋ฐฑ DAG์„ ๊ตฌ์ถ•ํ•˜๋Š” ๊ณผ์ •์„ ๋‹ค๋ฃน๋‹ˆ๋‹ค. ์„ ์ˆ˜์ง€์‹ MLOps ์šด์˜ ๊ณ ๋„ํ™” 2๋‹จ๊ณ„: Slack Alert ํ†ตํ•ฉ โ€” ํŒŒ์ดํ”„๋ผ์ธ ์•Œ๋ฆผ ์ž๋™ํ™” ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ ๋ชจ๋ธ ๋ฐฐํฌ๋Š” ์„ฑ๊ณตํ–ˆ์„ ๋•Œ๋ณด๋‹ค, ์‹คํŒจํ–ˆ์„ ๋•Œ์˜ ํšŒ๋ณต ์†๋„๊ฐ€ ์šด์˜ ํ’ˆ์งˆ์„ ๊ฒฐ์ •ํ•œ๋‹ค. ๋“ฑ๋ก์ด ์‹คํŒจํ•˜๋Š” ์ฆ‰์‹œ ์ด์ „ ์•ˆ์ • ๋ฒ„์ „์œผ๋กœ ๋Œ์•„๊ฐ€ FastAPI์— ์ž๋™ ๋ฐ˜์˜๋˜๋Š” ์™„์ „ํ•œ ๋กค๋ฐฑ ๋ฃจํ”„๋ฅผ ๊ตฌ์ถ•ํ•ด์•ผ ํ•œ๋‹ค. ์ด ๊ตฌ์กฐ๊ฐ€ ์žˆ์–ด์•ผ ์žฅ์•  ์ƒํ™ฉ์—์„œ๋„ ์„œ๋น„์Šค๊ฐ€ ๋Š๊ธฐ์ง€ ์•Š๊ณ  ๋ฐ”๋กœ ๋ณต๊ตฌ๋  ์ˆ˜ ์žˆ๋‹ค. ...

August 1, 2025 ยท 4 min

[MLOps ์šด์˜ ๊ณ ๋„ํ™” - 2๋‹จ๊ณ„: Slack Alert ํ†ตํ•ฉ (FastAPIยทAirflow ๊ณต์šฉ)]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ FastAPI์™€ Airflow๊ฐ€ ํฉ์–ด์ ธ ๋ณด๋‚ด๋˜ ์•Œ๋ฆผ์„ ํ•˜๋‚˜์˜ Slack ์ธํ„ฐํŽ˜์ด์Šค๋กœ ํ†ตํ•ฉํ•˜๊ณ , SealedSecret ๊ธฐ๋ฐ˜์œผ๋กœ Webhook URL์„ ์•ˆ์ „ํ•˜๊ฒŒ ์ฃผ์ž…ํ•˜๋Š” ๊ตฌ์กฐ๋ฅผ ๋‹ค๋ฃน๋‹ˆ๋‹ค. ์„ ์ˆ˜์ง€์‹ MLOps ์šด์˜ ๊ณ ๋„ํ™” 1๋‹จ๊ณ„: ํ•ซ์Šค์™‘ ๊ณ ๋„ํ™” โ€” /reload ๋ณด์•ˆ ๊ฐ•ํ™”์™€ DAG ์ž๋™ํ™” ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ ์šด์˜ ํ™˜๊ฒฝ์—์„œ ๊ฐ€์žฅ ๋ฌด์„œ์šด ๊ฑด ๋ฌธ์ œ๊ฐ€ ์ƒ๊ฒผ๋Š”๋ฐ ์•„๋ฌด๋„ ๋ชจ๋ฅด๊ณ  ์žˆ๋Š” ์ƒํ™ฉ์ด๋‹ค. FastAPI์™€ Airflow๊ฐ€ ๊ฐ์ž ํฉ์–ด์ ธ ๋ณด๋‚ด๋˜ ์•Œ๋ฆผ์„ ํ•˜๋‚˜์˜ Slack ์ธํ„ฐํŽ˜์ด์Šค๋กœ ์™„์ „ํžˆ ํ†ตํ•ฉํ•ด์„œ, ๋ชจ๋ธ ํ•™์Šตยท๋“ฑ๋กยท์„ผ์„œยทํ•ซ์Šค์™‘ยทํ—ฌ์Šค์ฒดํฌ๊นŒ์ง€ ์–ด๋””์—์„œ ๋ฌด์Šจ ์ผ์ด ์ผ์–ด๋‚˜๋„ ์ฆ‰์‹œ ๋ˆˆ์— ๋“ค์–ด์˜ค๋„๋ก ๋งŒ๋“ ๋‹ค. ...

July 26, 2025 ยท 4 min

[MLOps ์šด์˜ ๊ณ ๋„ํ™” - 1๋‹จ๊ณ„: ํ•ซ์Šค์™‘ ๊ณ ๋„ํ™” (/reload ๋ณด์•ˆยทDAG ์ž๋™ํ™”)]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ /reload ์—”๋“œํฌ์ธํŠธ์— 3์ค‘ ๋ณด์•ˆ(ํ† ํฐยทIP ํ™”์ดํŠธ๋ฆฌ์ŠคํŠธยทTLS)์„ ์ ์šฉํ•˜๊ณ , Airflow DAG์œผ๋กœ ํ•™์Šตโ†’๋“ฑ๋กโ†’๊ฐ์‹œโ†’ํ•ซ์Šค์™‘ ์ „์ฒด ์‹œํ€€์Šค๋ฅผ ์ž๋™ํ™”ํ•˜๋Š” ๊ณผ์ •์„ ๋‹ค๋ฃน๋‹ˆ๋‹ค. ์„ ์ˆ˜์ง€์‹ MLOps ์šด์˜ ๊ณ ๋„ํ™” 0๋‹จ๊ณ„: FastAPI A/BยทCanaryยทBlue-Green ์„œ๋น™ ๋ฒ ์ด์Šค โ€” ๋ฉ€ํ‹ฐ ๋ชจ๋ธ ๋ผ์šฐํŒ… ๊ธฐ๋ฐ˜ ์„ค๊ณ„ ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ ๋ชจ๋ธ์„ ์ž˜ ํ•™์Šต์‹œํ‚ค๋Š” ๊ฒƒ๋„ ์ค‘์š”ํ•˜์ง€๋งŒ, ์ž˜๋ชป๋œ ๋ชจ๋ธ์ด ์‹ค์ˆ˜๋กœ ํ•ซ์Šค์™‘๋˜๋Š” ์ˆœ๊ฐ„์ด ๋” ์น˜๋ช…์ ์ด๋‹ค. /reload๋ฅผ ํ™•์‹คํ•˜๊ฒŒ ์ž ๊ทธ๊ณ , ํ•™์Šตโ†’๋“ฑ๋กโ†’๊ฐ์‹œโ†’ํ•ซ์Šค์™‘๊นŒ์ง€๊ฐ€ Airflow์—์„œ ์ž๋™์œผ๋กœ ํ˜๋Ÿฌ๊ฐ€๋„๋ก ๋งŒ๋“ค์–ด์•ผ ํ•œ๋‹ค. ์ด ๋‹จ๊ณ„๊ฐ€ ์™„์„ฑ๋ผ์•ผ ์ดํ›„ ๋‹จ๊ณ„์—์„œ ์•ˆ์ „ํ•œ ์ž๋™ ์šด์˜(MLOps)์„ ๊ตฌํ˜„ํ•  ์ˆ˜ ์žˆ๋‹ค. ...

July 22, 2025 ยท 4 min

[MLOps ์šด์˜ ๊ณ ๋„ํ™” - 0๋‹จ๊ณ„: FastAPI A/BยทCanaryยทBlue-Green ์„œ๋น™ ๋ฒ ์ด์Šค]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ MLflow Alias ๊ธฐ๋ฐ˜์œผ๋กœ A/BยทCanaryยทBlue-Green ์„ธ ๊ฐ€์ง€ ์„œ๋น™ ์ „๋žต์„ ํ•˜๋‚˜์˜ FastAPI ์•ฑ์—์„œ ์ฒ˜๋ฆฌํ•˜๋Š” ๊ตฌ์กฐ๋ฅผ ์„ค๊ณ„ํ•˜๊ณ , ์ดํ›„ ์šด์˜ ์ž๋™ํ™”์˜ ๊ณตํ†ต ๋ฒ ์ด์Šค๋ฅผ ๋งŒ๋“œ๋Š” ๊ณผ์ •์„ ๋‹ค๋ฃน๋‹ˆ๋‹ค. ์„ ์ˆ˜์ง€์‹ [TS] Airflow ๊ธฐ์ดˆ ์ž๋™ํ™” ํŠธ๋Ÿฌ๋ธ”์ŠˆํŒ… โ€” Airflow โ†’ MLflow โ†’ FastAPI ์—ฐ๋™ ๊ธฐ๋ณธ ํ๋ฆ„ Level 3์€ ์ด ๊ธ€๋ถ€ํ„ฐ ์‹œ์ž‘ํ•ฉ๋‹ˆ๋‹ค. ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ ์šด์˜ ํ™˜๊ฒฝ์—์„œ ๋ชจ๋ธ์€ ์–ธ์ œ๋“  ๊ต์ฒด๋  ์ˆ˜ ์žˆ๊ณ , ๊ทธ ์ˆœ๊ฐ„์ด ์„œ๋น„์Šค ํ’ˆ์งˆ์ด ๊ฐ€์žฅ ํฌ๊ฒŒ ํ”๋“ค๋ฆฌ๋Š” ์œ„ํ—˜ ๊ตฌ๊ฐ„์ด๋‹ค. ์ดํ›„ ๋ชจ๋“  ์ž๋™ํ™”(ํ•™์Šตยท๋“ฑ๋กยทํ•ซ์Šค์™‘ยท๋กค๋ฐฑ)์˜ ๊ธฐ๋ฐ˜์ด ๋˜๋Š” A/BยทCanaryยทBlue-Green ์ „๋žต์„ ๋ชจ๋‘ ์ฒ˜๋ฆฌํ•  ์ˆ˜ ์žˆ๋Š” FastAPI ์„œ๋น™ ๊ตฌ์กฐ๋ถ€ํ„ฐ ์žก์•„์•ผ ํ•œ๋‹ค. ์ด ๋ผˆ๋Œ€๊ฐ€ ์™„์„ฑ๋ผ์•ผ Airflow, MLflow, ArgoCD์™€ ์—ฐ๊ฒฐ๋œ ์šด์˜ํ˜• MLOps ํŒŒ์ดํ”„๋ผ์ธ์„ ๋งŒ๋“ค ์ˆ˜ ์žˆ๋‹ค. ...

July 18, 2025 ยท 5 min

[MLOps ํ”Œ๋žซํผ ๊ตฌ์ถ• : Airflow-MLflow-FastAPI (Helm)]

๐Ÿงฉ ์‹ค์ „ ์‹œ๋‚˜๋ฆฌ์˜ค ๊ธฐ๋ฐ˜ ๊ตฌ์„ฑ ๋ฐฐ๊ฒฝ ์ด ํ”„๋กœ์ ํŠธ๋Š” ๋‹จ์ˆœ ์‹ค์Šต์„ ๋„˜์–ด์„œ, ์‹ค์ œ ๋ฐœ์ƒํ•˜๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๋ฌธ์ œ๋“ค์„ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•œ MLOps ์ธํ”„๋ผ ๊ตฌ์ถ•์„ ๋ชฉํ‘œ๋กœ ์„ค๊ณ„๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ์—ฌ๋Ÿฌ ๋ชจ๋ธ ์‹คํ—˜ ๊ฒฐ๊ณผ๊ฐ€ ๋’ค์„ž์—ฌ ์ถ”์ ์ด ์–ด๋ ค์šด ๋ฌธ์ œ โ†’ MLflow Tracking ์„œ๋ฒ„ + PostgreSQL ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ์ €์žฅ์†Œ ๊ตฌ์„ฑ ๋ชจ๋ธ ํŒŒ์ผ ๋ฐ ๋กœ๊ทธ๊ฐ€ ๋กœ์ปฌ์—๋งŒ ์ €์žฅ๋˜์–ด ํ˜‘์—… ๋ฐ ์žฌํ˜„์„ฑ์ด ๋–จ์–ด์ง€๋Š” ๋ฌธ์ œ โ†’ S3 ๊ธฐ๋ฐ˜ artifact store ๊ตฌ์„ฑ + pyfunc ๊ธฐ๋ฐ˜ ๋ชจ๋ธ ์„œ๋น™ ๊ตฌ์กฐ ์„ค๊ณ„ ์ˆ˜์ž‘์—… DAG ๋“ฑ๋ก, ๋ชจ๋ธ ๋ฐฐํฌ ๋“ฑ์˜ ๋น„ํšจ์œจ์  ์šด์˜ ๋ฌธ์ œ โ†’ Airflow + GitSync ์—ฐ๋™์œผ๋กœ ํŒŒ์ดํ”„๋ผ์ธ ์ž๋™ํ™” ๋ฐ ๋ฒ„์ „ ๊ด€๋ฆฌ ๊ฐ€๋Šฅ ...

July 15, 2025 ยท 3 min

[MLOps ํ”Œ๋žซํผ ๊ตฌ์ถ• - 6๋‹จ๊ณ„: ์‹ค์‹œ๊ฐ„ ๋ชจ๋ธ ํ•ซ์Šค์™‘ ๊ตฌ์กฐ ์‹คํ—˜]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ Airflow DAG์—์„œ ์กฐ๊ฑด๋ถ€ ๋ชจ๋ธ ๋“ฑ๋ก ํ›„ FastAPI ํ•ซ์Šค์™‘๊นŒ์ง€ E2E ์ž๋™ํ™” ํ๋ฆ„์„ ์‹คํ—˜ํ•ฉ๋‹ˆ๋‹ค. ์„ ์ˆ˜์ง€์‹ MLOps ํ”Œ๋žซํผ ๊ตฌ์ถ• 5๋‹จ๊ณ„: FastAPI ์„œ๋น™ ๋ฐ ํ•ซ์Šค์™‘ ๊ตฌ์กฐ ๊ตฌ์ถ• โ€” FastAPI ๋ชจ๋ธ ๋กœ๋”ฉ๊ณผ /reload API ๊ตฌ์กฐ ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ ๊ฐœ๋ณ„ ์ปดํฌ๋„ŒํŠธ(Airflow, MLflow, FastAPI)๋ฅผ ๊ฐ๊ฐ ๊ตฌ์„ฑํ–ˆ์ง€๋งŒ, ํ•™์Šต๋ถ€ํ„ฐ ์„œ๋น™๊นŒ์ง€์˜ ์ž๋™ํ™” ํ๋ฆ„์„ E2E๋กœ ๊ฒ€์ฆํ•ด์•ผ ์‹ค์ œ ์šด์˜ ๊ฐ€๋Šฅ ์—ฌ๋ถ€๋ฅผ ํŒ๋‹จํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด ๋‹จ๊ณ„์—์„œ๋Š” Airflow DAG์—์„œ ๋ชจ๋ธ ํ•™์Šต โ†’ ์„ฑ๋Šฅ ๊ธฐ์ค€ ๋ถ„๊ธฐ โ†’ MLflow ๋“ฑ๋ก โ†’ FastAPI ํ•ซ์Šค์™‘๊นŒ์ง€ ์ „์ฒด ํŒŒ์ดํ”„๋ผ์ธ์„ ์‹คํ—˜ํ•ฉ๋‹ˆ๋‹ค. ...

July 10, 2025 ยท 2 min

[MLOps ํ”Œ๋žซํผ ๊ตฌ์ถ• - 4๋‹จ๊ณ„: Airflow : GitSync + ์™ธ๋ถ€ PostgreSQL + Secret ์—ฐ๋™]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ Airflow๋ฅผ Helm์œผ๋กœ ๋ฐฐํฌํ•˜๋ฉด์„œ GitSync DAG ๋™๊ธฐํ™”, ์™ธ๋ถ€ PostgreSQL, AWS Secret ๋งˆ์šดํŠธ๋ฅผ ์—ฐ๋™ํ•ฉ๋‹ˆ๋‹ค. ์„ ์ˆ˜์ง€์‹ MLOps ํ”Œ๋žซํผ ๊ตฌ์ถ• 3๋‹จ๊ณ„: MLflow Helm ๊ตฌ์„ฑ โ€” Helm ๊ธฐ๋ฐ˜ ๋ฐฐํฌ์™€ Secret ์ฃผ์ž… ํŒจํ„ด ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ DAG ์ฝ”๋“œ๋ฅผ ์ˆ˜๋™์œผ๋กœ ์—…๋กœ๋“œํ•˜๋ฉด ๋ฒ„์ „ ๊ด€๋ฆฌ์™€ ํ˜‘์—…์ด ๋ถˆ๊ฐ€๋Šฅํ•ฉ๋‹ˆ๋‹ค. Git ๊ธฐ๋ฐ˜ DAG ๊ด€๋ฆฌ(GitSync)๋ฅผ ๋„์ž…ํ•˜๊ณ , SSH Key/AWS ์ธ์ฆ์„ Secret์œผ๋กœ ์ฃผ์ž…ํ•˜๋ฉฐ, ์™ธ๋ถ€ PostgreSQL์„ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ์ €์žฅ์†Œ๋กœ ์—ฐ๋™ํ•˜๋Š” GitOps ํŒจํ„ด์„ ๊ตฌ์„ฑํ•ฉ๋‹ˆ๋‹ค. ๐Ÿ“ ์•„ํ‚คํ…์ฒ˜ ๊ตฌ์„ฑ๋„ ๐Ÿณ ์ปค์Šคํ…€ Airflow ์ด๋ฏธ์ง€ ๊ตฌ์„ฑ GitSync DAG์—์„œ MLflow ์—ฐ๋™ ๋˜๋Š” AWS SDK ์‚ฌ์šฉ์„ ์œ„ํ•œ Python ํŒจํ‚ค์ง€ ์„ค์น˜๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค. ...

July 3, 2025 ยท 3 min

[Airflow ๊ธฐ์ดˆ ์ž๋™ํ™” - Airflow โ†’ MLflow โ†’ FastAPI]

๐Ÿงญ ์ „์ฒด ํ๋ฆ„ ์˜ˆ์‹œ [AIRFLOW DAG ์‹คํ–‰] โ†“ [train_mlflow.py] - iris ๋ชจ๋ธ ํ•™์Šต - ํŒŒ๋ผ๋ฏธํ„ฐ/๋ฉ”ํŠธ๋ฆญ ๋กœ๊น… - ๋ชจ๋ธ Registry ๋“ฑ๋ก โ†“ [promote_mlflow.py] - ์ตœ์‹  ๋ชจ๋ธ์„ Production์œผ๋กœ ์ „ํ™˜ โ†“ [FastAPI] - models:/IrisModel/Production โ†’ ์‹ค์‹œ๊ฐ„ ์˜ˆ์ธก ๐Ÿ‘‰ ์‹ค์Šต ์ฝ”๋“œ๋Š” ๐Ÿ”— GitHub (Airflow + MLflow + FastAPI) โœ… [1๋‹จ๊ณ„] ํ”„๋กœ์ ํŠธ ๊ธฐ๋ณธ ํด๋” ๊ตฌ์กฐ ์„ค๊ณ„ ๐Ÿ“ 1. ์ „์ฒด ๋””๋ ‰ํ† ๋ฆฌ ๊ตฌ์„ฑ๋„ mlops_project/ โ”œโ”€โ”€ airflow/ ๐Ÿ›ซ Airflow ์„ค์ • ๋ฐ DAG ์Šค์ผ€์ค„๋Ÿฌ โ”‚ โ”œโ”€โ”€ dags/ โ† DAG ์ •์˜ ๋””๋ ‰ํ† ๋ฆฌ โ”‚ โ”‚ โ””โ”€โ”€ train_with_mlflow.py โ† ํ•™์Šต DAG (MLflow ์—ฐ๋™) โ”‚ โ”œโ”€โ”€ Dockerfile.airflow โ† Airflow์šฉ Dockerfile โ”‚ โ”œโ”€โ”€ requirements.txt โ† Airflow ์˜์กด์„ฑ โ”‚ โ””โ”€โ”€ .dockerignore โ”‚ โ”œโ”€โ”€ fastapi/ โšก FastAPI ์˜ˆ์ธก API ์„œ๋ฒ„ โ”‚ โ”œโ”€โ”€ app/ โ”‚ โ”‚ โ””โ”€โ”€ main.py โ† ๋ชจ๋ธ ์„œ๋น™ ์—”๋“œํฌ์ธํŠธ โ”‚ โ”œโ”€โ”€ Dockerfile.api โ† FastAPI์šฉ Dockerfile โ”‚ โ”œโ”€โ”€ requirements.txt โ† FastAPI ์˜์กด์„ฑ โ”‚ โ””โ”€โ”€ .dockerignore โ”‚ โ”œโ”€โ”€ ml_code/ ๐Ÿง  ML ํ•™์Šต ๋ฐ ํ”„๋กœ๋ชจ์…˜ ์ฝ”๋“œ โ”‚ โ”œโ”€โ”€ train_mlflow.py โ† ๋ชจ๋ธ ํ•™์Šต ๋ฐ MLflow ๋กœ๊น… โ”‚ โ””โ”€โ”€ promote_mlflow.py โ† ๋ชจ๋ธ ํ”„๋กœ๋ชจ์…˜ (Staging โ†’ Production) โ”‚ โ”œโ”€โ”€ mlflow_store/ ๐Ÿ—‚๏ธ MLflow ์ €์žฅ์†Œ ๊ฒฝ๋กœ (๋ณผ๋ฅจ) โ”‚ โ”œโ”€โ”€ Dockerfile.mlflow โ† MLflow ์„œ๋ฒ„ ์ปค์Šคํ„ฐ๋งˆ์ด์ง• โ”‚ โ”œโ”€โ”€ mlflow.db โ† Model Registry DB (sqlite) โ”‚ โ”œโ”€โ”€ mlruns/ โ† ์‹คํ—˜ ๋กœ๊ทธ ๋””๋ ‰ํ† ๋ฆฌ โ”‚ โ”œโ”€โ”€ artifacts/ โ† ๋ชจ๋ธ ์•„ํ‹ฐํŒฉํŠธ ์ €์žฅ์†Œ โ”‚ โ””โ”€โ”€ .dockerignore โ”‚ โ”œโ”€โ”€ docker-compose.yaml ๐Ÿงฉ ์ „์ฒด ์„œ๋น„์Šค ๊ตฌ์„ฑ ์ •์˜ โ”œโ”€โ”€ .env ๐Ÿ” ๋ฏผ๊ฐ ์ •๋ณด (.env๋กœ ๋ถ„๋ฆฌ) โ”œโ”€โ”€ README.md ๐Ÿ“ ์ „์ฒด ํ”„๋กœ์ ํŠธ ๋ฌธ์„œํ™” โ”œโ”€โ”€ .gitignore โ””โ”€โ”€ .dockerignore โœ… [2๋‹จ๊ณ„] docker-compose.yaml ํ†ตํ•ฉ ๊ตฌ์„ฑ ๐Ÿงญ ๊ตฌ์„ฑ ๋ชฉํ‘œ ์„œ๋น„์Šค๋ช… ์„ค๋ช… ํฌํŠธ airflow DAG ์‹คํ–‰ ํ™˜๊ฒฝ (webserver/scheduler) 8080 postgres Airflow ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ์ €์žฅ์šฉ DB ๋‚ด๋ถ€ ํ†ต์‹  mlflow MLflow UI + Registry ๊ธฐ๋Šฅ 5000 fastapi ์ถ”๋ก  API ์„œ๋ฒ„ (๋ชจ๋ธ ๋กœ๋”ฉ) 8000 ์ด๋ฏธ์ง€ ์‚ฌ์šฉ์‹œ ์ฃผ์˜ (UI๋งŒ ์ œ๊ณตํ•˜๋Š” ์ด๋ฏธ์ง€ ์กด์žฌ) ๐Ÿ“„ docker-compose.yaml ์ „์ฒด ์˜ˆ์‹œ version: '3.8' services: # ๐Ÿ“ฆ PostgreSQL: Airflow ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ์ €์žฅ์šฉ DB postgres: image: postgres:13 container_name: postgres env_file: - .env # โ† ๋ฏผ๊ฐ์ •๋ณด ๋ถ„๋ฆฌ (์•„์ด๋””/๋น„๋ฒˆ) environment: POSTGRES_USER: ${POSTGRES_USER} POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} POSTGRES_DB: ${POSTGRES_DB} volumes: # โ† ์ฝ”๋“œ/๋ฐ์ดํ„ฐ ๊ณต์œ  ๋ฐ ์˜์†์„ฑ ๋ณด์žฅ - postgres_data:/var/lib/postgresql/data # โ† DB ๋ฐ์ดํ„ฐ ์œ ์ง€ (์žฌ์‹œ์ž‘ ๋Œ€๋น„) # ๐Ÿ›ซ Airflow: DAG ์Šค์ผ€์ค„๋Ÿฌ ๋ฐ ํƒœ์Šคํฌ ์‹คํ–‰ airflow: build: context: ./airflow # โ†’ Airflow ์ „์šฉ Dockerfile ๊ฒฝ๋กœ dockerfile: Dockerfile.airflow container_name: airflow command: standalone # โ†’ ๋กœ์ปฌ ํ…Œ์ŠคํŠธ์šฉ ๊ฐ„๋‹จ ์‹คํ–‰ ๋ช…๋ น # (- Scheduler + Webserver + DB ์ดˆ๊ธฐํ™”๊นŒ์ง€ ์ž๋™์œผ๋กœ ํ•œ๋ฒˆ์— ์‹คํ–‰) # (- ์‹ค๋ฌด/์šด์˜์—์„œ๋Š” airflow-webserver, airflow-scheduler ํ•„๋“œ ๋ถ„๋ฆฌ) ports: - "8080:8080" # โ†’ Airflow ์›น UI (localhost:8080) depends_on: - postgres # โ†’ DB๊ฐ€ ๋จผ์ € ์˜ฌ๋ผ์™€์•ผ Airflow ์‹œ์ž‘ ๊ฐ€๋Šฅ env_file: - .env environment: # Airflow ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ DB ์—ฐ๊ฒฐ ์ฃผ์†Œ AIRFLOW__CORE__SQL_ALCHEMY_CONN: ${AIRFLOW__CORE__SQL_ALCHEMY_CONN} # Airflow ์˜ˆ์ œ DAG ๋ถˆ๋Ÿฌ์˜ฌ์ง€ ์—ฌ๋ถ€ AIRFLOW__CORE__LOAD_EXAMPLES: ${AIRFLOW__CORE__LOAD_EXAMPLES} MLFLOW_TRACKING_URI: http://mlflow:5000 # โ†’ DAG ์ฝ”๋“œ์—์„œ MLflow ์—ฐ๋™ volumes: - ./airflow/dags:/opt/airflow/dags # DAG ํŒŒ์ผ mount - ./ml_code:/opt/airflow/ml_code # ํ•™์Šต ์ฝ”๋“œ ๊ณต์œ  - ./mlflow_store:/mlflow # ๋ชจ๋ธ ์ €์žฅ์†Œ ๊ณต์œ  # ๐Ÿ”ฌ MLflow: ์‹คํ—˜ ์ถ”์  + ๋ชจ๋ธ ๋ ˆ์ง€์ŠคํŠธ๋ฆฌ ์„œ๋ฒ„ mlflow: build: context: ./mlflow_store # ์ปค์Šคํ…€ Dockerfile ์œ„์น˜ dockerfile: Dockerfile.mlflow ports: - "5000:5000" # โ†’ MLflow UI (localhost:5000) volumes: - ./mlflow_store:/mlflow # ์‹คํ—˜ ๋กœ๊ทธ + DB + artifacts ์ €์žฅ environment: - MLFLOW_TRACKING_URI=http://0.0.0.0:5000 # ๋‚ด๋ถ€ ์ปจํ…Œ์ด๋„ˆ ๊ธฐ์ค€ URI # โšก FastAPI: ๋ชจ๋ธ ์„œ๋น™ API fastapi: build: context: ./fastapi dockerfile: Dockerfile.api container_name: fastapi ports: - "8000:8000" # โ†’ ์˜ˆ์ธก API ์—”๋“œํฌ์ธํŠธ (localhost:8000) volumes: - ./fastapi/app:/app/app # FastAPI app ๋””๋ ‰ํ† ๋ฆฌ mount - ./ml_code:/app/ml_code # ํ•™์Šต/๋ชจ๋ธ ์ฝ”๋“œ ๊ณต์œ  - ./mlflow_store:/mlflow # ์ €์žฅ๋œ ๋ชจ๋ธ ๋ถˆ๋Ÿฌ์˜ค๊ธฐ ์œ„ํ•œ mount # ๐Ÿ—‚๏ธ ๋ณผ๋ฅจ ์ •์˜ (Postgres DB ์˜์†์„ฑ ์œ ์ง€) volumes: postgres_data: ๐ŸŽ ์ถ”๊ฐ€๋กœ ํ•ด์•ผ ํ•  ๊ฒƒ Airflow ์ฒซ ์‹คํ–‰ ํ›„์—” ๋ณดํ†ต ๊ด€๋ฆฌ์ž ๊ณ„์ • ์ƒ์„ฑ๋„ ํ•ด์ค˜์•ผ ํ•จ: # airflow ์ปจํ…Œ์ด๋„ˆ ์ ‘์† docker exec -it airflow bash # ๊ด€๋ฆฌ์ž ๊ณ„์ • ์ƒ์„ฑ airflow users create \ --username airflow \ --password airflow \ --firstname Keoho \ --lastname Ban \ --role Admin \ --email airflow@example.com ๐Ÿ” [๊ตฌ์ถ• Tip] Airflow, FastAPI, MLflow ๊ฐ„ ๊ณต์œ  ๋ณผ๋ฅจ ๊ตฌ์กฐ ํ™•์ธ ๊ณต์œ  ๋ฆฌ์†Œ์Šค ์„ค๋ช… ./mlflow_store:/mlflow (MLflow) MLflow ์„œ๋ฒ„๊ฐ€ ์“ฐ๋Š” ๋กœ๊ทธ/๋ชจ๋ธ ์ €์žฅ์†Œ ./mlflow_store:/mlflow (Airflow) ํ•™์Šต ํ›„ ๋ชจ๋ธ ์ €์žฅ ์œ„์น˜ ๊ณต์œ  ./mlflow_store:/mlflow (FastAPI) ๋ชจ๋ธ ์ถ”๋ก  ์‹œ ๋กœ๋“œ ๊ฒฝ๋กœ ๊ณต์œ  โžก ๊ฒฝ๋กœ ํ†ต์ผ์„ฑ์ด ์ค‘์š”ํ•จ! ์ง€๊ธˆ์€ ๋ชจ๋‘ ./mlflow๋กœ ๊ณต์œ  (./mlflow ํ•˜์œ„์— /mlruns ์กด์žฌ) ...

June 13, 2025 ยท 8 min