[MLOps ํ”Œ๋žซํผ ๊ตฌ์ถ• - 4๋‹จ๊ณ„: Airflow : GitSync + ์™ธ๋ถ€ PostgreSQL + Secret ์—ฐ๋™]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ Airflow๋ฅผ Helm์œผ๋กœ ๋ฐฐํฌํ•˜๋ฉด์„œ GitSync DAG ๋™๊ธฐํ™”, ์™ธ๋ถ€ PostgreSQL, AWS Secret ๋งˆ์šดํŠธ๋ฅผ ์—ฐ๋™ํ•ฉ๋‹ˆ๋‹ค. ์„ ์ˆ˜์ง€์‹ MLOps ํ”Œ๋žซํผ ๊ตฌ์ถ• 3๋‹จ๊ณ„: MLflow Helm ๊ตฌ์„ฑ โ€” Helm ๊ธฐ๋ฐ˜ ๋ฐฐํฌ์™€ Secret ์ฃผ์ž… ํŒจํ„ด ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ DAG ์ฝ”๋“œ๋ฅผ ์ˆ˜๋™์œผ๋กœ ์—…๋กœ๋“œํ•˜๋ฉด ๋ฒ„์ „ ๊ด€๋ฆฌ์™€ ํ˜‘์—…์ด ๋ถˆ๊ฐ€๋Šฅํ•ฉ๋‹ˆ๋‹ค. Git ๊ธฐ๋ฐ˜ DAG ๊ด€๋ฆฌ(GitSync)๋ฅผ ๋„์ž…ํ•˜๊ณ , SSH Key/AWS ์ธ์ฆ์„ Secret์œผ๋กœ ์ฃผ์ž…ํ•˜๋ฉฐ, ์™ธ๋ถ€ PostgreSQL์„ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ์ €์žฅ์†Œ๋กœ ์—ฐ๋™ํ•˜๋Š” GitOps ํŒจํ„ด์„ ๊ตฌ์„ฑํ•ฉ๋‹ˆ๋‹ค. ๐Ÿ“ ์•„ํ‚คํ…์ฒ˜ ๊ตฌ์„ฑ๋„ ๐Ÿณ ์ปค์Šคํ…€ Airflow ์ด๋ฏธ์ง€ ๊ตฌ์„ฑ GitSync DAG์—์„œ MLflow ์—ฐ๋™ ๋˜๋Š” AWS SDK ์‚ฌ์šฉ์„ ์œ„ํ•œ Python ํŒจํ‚ค์ง€ ์„ค์น˜๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค. ...

July 3, 2025 ยท 3 min

[MLOps ํ”Œ๋žซํผ ๊ตฌ์ถ• - 3๋‹จ๊ณ„: MLflow : PostgreSQL + S3 ์—ฐ๋™ ๊ธฐ๋ฐ˜ Helm ๊ตฌ์„ฑ]

์ด ๊ธ€์—์„œ ๋‹ค๋ฃจ๋Š” ๊ฒƒ MLflow๋ฅผ PostgreSQL + S3 ๋ฐฑ์—”๋“œ๋กœ ๊ตฌ์„ฑํ•˜์—ฌ Kubernetes์— Helm ๋ฐฐํฌํ•˜๊ณ , ์ปค์Šคํ…€ ์ด๋ฏธ์ง€์™€ Ingress๋ฅผ ์—ฐ๋™ํ•ฉ๋‹ˆ๋‹ค. ์„ ์ˆ˜์ง€์‹ MLOps ํ”Œ๋žซํผ ๊ตฌ์ถ• 2๋‹จ๊ณ„: S3 & PostgreSQL Secret ๊ด€๋ฆฌ โ€” Secret/ConfigMap ์ฃผ์ž… ์ „๋žต ์ด ๋‹จ๊ณ„์—์„œ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ๋ฌธ์ œ ํ•œ ํ”„๋กœ์ ํŠธ์— ์—ฌ๋Ÿฌ ๋ฒ„์ „์˜ ๋ชจ๋ธ์ด ๋“ฑ์žฅํ•˜๋ฉด, ๊ฐ ์‹คํ—˜์˜ ํŒŒ๋ผ๋ฏธํ„ฐ/๋ฉ”ํŠธ๋ฆญ/์•„ํ‹ฐํŒฉํŠธ๋ฅผ ์ฒด๊ณ„์ ์œผ๋กœ ์ถ”์ ํ•  ์„œ๋ฒ„๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค. ๋กœ์ปฌ MLflow UI(SQLite + file ์ €์žฅ)๋กœ๋Š” ํŒ€ ๊ณต์œ ์™€ ์•„ํ‹ฐํŒฉํŠธ ์˜์†์„ฑ์ด ๋ถ€์กฑํ•ฉ๋‹ˆ๋‹ค. ์ด ๋‹จ๊ณ„์—์„œ๋Š” MLflow๋ฅผ PostgreSQL(๋ฉ”ํƒ€๋ฐ์ดํ„ฐ) + S3(์•„ํ‹ฐํŒฉํŠธ) ๋ฐฑ์—”๋“œ๋กœ ๊ตฌ์„ฑํ•˜๊ณ  Helm์œผ๋กœ Kubernetes์— ๋ฐฐํฌํ•ฉ๋‹ˆ๋‹ค. ...

June 30, 2025 ยท 3 min