Feature Store๋Š” โ€˜ํ”ผ์ฒ˜ ์ƒ์„ฑโ€™์ด ์•„๋‹ˆ๋ผ ์šด์˜ ์žฌํ˜„์„ฑ์˜ ๊ธฐ์ค€


๐Ÿ“Œ ์‹œ๋ฆฌ์ฆˆ ์ „์ฒด ๊ฒฝ๋กœ ์š”์•ฝ

์ˆœ์„œ์ฃผ์ œ
1Contract ๊ธฐ๋ฐ˜ Feature Store-lite ๊ตฌ์ถ•
2Feast ํ™•์žฅ์œผ๋กœ ์กฐํšŒ ๊ฐ€๋Šฅํ•œ ํ”Œ๋žซํผ ์™„์„ฑ
3Proof๋กœ ์šด์˜ ์‹ ๋ขฐ์„ฑ ๊ฒ€์ฆ

์ด ์‹œ๋ฆฌ์ฆˆ๋Š” โ€œFeast๋ฅผ ์จ๋ดค๋‹คโ€๊ฐ€ ์•„๋‹ˆ๋ผ,

Feature Store๋ฅผ ์šด์˜ ๊ฐ€๋Šฅํ•œ ์‹œ์Šคํ…œ์œผ๋กœ ๊ณ ์ •ํ•˜๋Š” ๊ณผ์ •์„ ๊ธฐ๋กํ–ˆ์Šต๋‹ˆ๋‹ค.


๐ŸŽฏ ์ „์ฒด ํšŒ๊ณ  ์š”์•ฝ

๋‹จ๊ณ„๋ชฉํ‘œ๊ณ ์ •๋œ ๊ธฐ์ค€
1Feature Store-lite ๋ผˆ๋Œ€ ๊ตฌ์ถ•Contract ๋ถ„๋ฆฌ + ๋ฒ„์ „ 1๋‹จ + ์žฌํ˜„์„ฑ ๋ฉ”ํƒ€
2Feast๋กœ ํ”Œ๋žซํผ ํ™•์žฅversion + latest ์ •์ฑ… + offline/online ์ผ๊ด€์„ฑ
3Proof๋กœ ์‹ ๋ขฐ์„ฑ ํ™•๋ณดE2E ๊ฒ€์ฆ + ์žฅ์•  ์žฌํ˜„ยท๋ณต๊ตฌ

๐Ÿ”„ ์ด ์‹œ๋ฆฌ์ฆˆ๋ฅผ ๊ด€ํ†ตํ•˜๋Š” ํ•œ ๋ฌธ์žฅ

Contract โ†’ Build โ†’ Versioned Store โ†’ Latest Pointer โ†’ Apply โ†’ Materialize โ†’ Online Read

Feature Store๋ฅผ

  • ํŒŒ์ผ ์ƒ์„ฑ ํŒŒ์ดํ”„๋ผ์ธ์ด ์•„๋‹ˆ๋ผ
  • *์šด์˜ ๊ธฐ์ค€์ด ๊ณ ์ •๋œ ์ƒํƒœ ๋จธ์‹ (state machine)**์œผ๋กœ ์ •์˜ํ–ˆ๊ณ 

๋ชจ๋“  ๊ฒฐ๊ณผ๊ฐ€ ์žฌํ˜„ ๊ฐ€๋Šฅํ•œ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ๋กœ ๋‚จ๋„๋ก ์„ค๊ณ„ํ–ˆ์Šต๋‹ˆ๋‹ค.


๐Ÿงฉ Feature Store ์ „์ฒด ๊ตฌ์กฐ (Lite โ†’ Feast)

mermaid-feature-epilog-01.png

๊ตฌ์กฐ ์š”์•ฝ

  • Contract ๊ณ„์ธต

    schema / repo๋ฅผ GitOps ๋ฆฌ์†Œ์Šค๋กœ ๊ณ ์ •

  • Control ๊ณ„์ธต

    Airflow๋Š” โ€œ์ƒ์„ฑ + ์ €์žฅ ์ •์ฑ…โ€๊นŒ์ง€๋งŒ ๋‹ด๋‹น

  • Storage ๊ณ„์ธต

    version(์žฌํ˜„์„ฑ) + latest(์šด์˜ ํŽธ์˜) ๋™์‹œ ์œ ์ง€

  • Platform ๊ณ„์ธต

    Feast๊ฐ€ offline / online / registry๋ฅผ ์ฑ…์ž„

  • dev/prod ๋ถ„๋ฆฌ ์›์น™

    namespace ยท prefix ยท application ๋‹จ์œ„ ์™„์ „ ๋ถ„๋ฆฌ


๐Ÿ” Feature Store Operational Loop

mermaid-feature-epilog-02.png

  • GitOps๋กœ ๊ณ„์•ฝ์ด ๋จผ์ € ๊ณ ์ •๋˜๊ณ 
  • Airflow๋Š” ๊ณ„์•ฝ์„ ์ฝ์–ด ๋™์ผํ•œ ๊ทœ์น™์œผ๋กœ ํ”ผ์ฒ˜๋ฅผ ์ƒ์„ฑํ•˜๋ฉฐ
  • S3์—๋Š” version๊ณผ latest๊ฐ€ ํ•จ๊ป˜ ์œ ์ง€๋˜๊ณ 
  • Feast๋Š” latest๋งŒ ๋ฐ”๋ผ๋ณด๋ฉฐ offline/online ์ผ๊ด€์„ฑ์„ ๋ณด์žฅํ•ฉ๋‹ˆ๋‹ค

๋ชจ๋ธ ํ•™์Šต, ์„œ๋น™, ๋ชจ๋‹ˆํ„ฐ๋ง์€

feature version ๋˜๋Š” registry ๊ธฐ์ค€์œผ๋กœ ์ž์—ฐ์Šค๋Ÿฝ๊ฒŒ ํ™•์žฅ๋ฉ๋‹ˆ๋‹ค.


๐Ÿง  ์šด์˜ ์›์น™ ์ •๋ฆฌ

1๏ธโƒฃ Contract First

  • schema / repo๋Š” ์ฝ”๋“œ๊ฐ€ ์•„๋‹ˆ๋ผ ์šด์˜ ๋ฆฌ์†Œ์Šค
  • GitOps๋กœ ๋ฐฐํฌยท๋กค๋ฐฑ ๊ฐ€๋Šฅํ•ด์•ผ ํ•จ
  • ํŒŒ์ดํ”„๋ผ์ธ์€ โ€œ์ฝ๊ธฐ ์ „์šฉ ์†Œ๋น„์žโ€๋กœ ์ œํ•œ

2๏ธโƒฃ Version์€ ๊นŠ์–ด์ง€์ง€ ์•Š๋Š”๋‹ค

  • /<feature_set>/<version>/ 1๋‹จ ๊ณ ์ •
  • ์žฌํ˜„์„ฑ์€ depth๊ฐ€ ์•„๋‹ˆ๋ผ ๊ทœ์น™์—์„œ ๋‚˜์˜จ๋‹ค

3๏ธโƒฃ latest๋Š” ํŽธ์˜ ์žฅ์น˜์ผ ๋ฟ์ด๋‹ค

  • Feast offline์€ latest๋งŒ ์‚ฌ์šฉ
  • ์žฌํ˜„/๋ถ„์„/ํ•™์Šต์€ ์–ธ์ œ๋“  version์œผ๋กœ ํšŒ๊ท€ ๊ฐ€๋Šฅ

4๏ธโƒฃ ์‹คํŒจ๋Š” ์‚ญ์ œํ•˜์ง€ ์•Š๊ณ  ๊ฒฉ๋ฆฌํ•œ๋‹ค

  • s3fs, subPath, Airflow Variable SDK ๋“ฑ

    ์‹ค์ œ๋กœ ๊นจ์กŒ๋˜ ์ง€์ ์€

    ์žฌํ˜„ โ†’ ํ•ด๊ฒฐ โ†’ ๊ฐ€๋“œ๋ ˆ์ผ๋กœ ๊ณ ์ •

  • ์šด์˜ ์‹œ์Šคํ…œ์€ โ€œ๊นจ์กŒ๋˜ ํ”์ โ€์„ ๋‚จ๊ธธ์ˆ˜๋ก ๊ฐ•ํ•ด์ง„๋‹ค


โœ… ์ตœ์ข… ์ ๊ฒ€ ์ฒดํฌ๋ฆฌ์ŠคํŠธ (Proof ๊ธฐ๋ฐ˜)

  • ArgoCD์—์„œ feature-store / feast devยทprod Synced/Healthy
  • Airflow Pod์—์„œ ๊ณ„์•ฝ ํŒŒ์ผ mount ํ™•์ธ
  • DAG E2E ์„ฑ๊ณต (build โ†’ store)
  • S3์— version prefix + latest ๋™์‹œ ์กด์žฌ
  • metadata.json์— schema_hash / generated_at ๊ธฐ๋ก
  • Feast apply ์„ฑ๊ณต + registry S3 ๋ฐ˜์˜
  • materialize ์„ฑ๊ณต (s3fs ํฌํ•จ ์ด๋ฏธ์ง€)
  • online(redis) ์กฐํšŒ ๊ฒฐ๊ณผ๊ฐ€ None ์•„๋‹˜

๐Ÿ ํšŒ๊ณ 

์ด๋ฒˆ ์‹œ๋ฆฌ์ฆˆ์˜ ๊ฒฐ๋ก ์€

โ€œFeature Store๋ฅผ ๋งŒ๋“ค์—ˆ๋‹คโ€๊ฐ€ ์•„๋‹™๋‹ˆ๋‹ค.

ํ”ผ์ฒ˜๊ฐ€ ์ƒ์„ฑยท์ €์žฅยท์กฐํšŒ๋˜๋Š” ์ „ ๊ณผ์ •์„ โ€˜์šด์˜ ๊ธฐ์ค€โ€™์œผ๋กœ ๊ณ ์ •ํ–ˆ๋‹ค๋Š” ์ ์ž…๋‹ˆ๋‹ค.

Feature Store-lite์—์„œ๋Š” ์žฌํ˜„์„ฑ์˜ ๋ผˆ๋Œ€๋ฅผ ๋งŒ๋“ค์—ˆ๊ณ ,

Feast ํ™•์žฅ์—์„œ๋Š” ๊ทธ ์œ„์— ์กฐํšŒ ๊ฐ€๋Šฅํ•œ ํ”Œ๋žซํผ์„ ์˜ฌ๋ ธ์Šต๋‹ˆ๋‹ค.

ํŠนํžˆ, ์‹ค์ œ๋กœ ๊นจ์กŒ๋˜ ํฌ์ธํŠธ๋“ค์„ ํšŒํ”ผํ•˜์ง€ ์•Š๊ณ 

์žฌํ˜„ โ†’ ํ•ด๊ฒฐ โ†’ ๊ตฌ์กฐ ๊ณ ์ •๊นŒ์ง€ ๊ฐ€์ ธ๊ฐ”์Šต๋‹ˆ๋‹ค.

์ด์ œ ์ด Feature Store๋Š”

Training / Serving / Monitoring์œผ๋กœ

์ž์—ฐ์Šค๋Ÿฝ๊ฒŒ ํ™•์žฅ๋  ์ˆ˜ ์žˆ๋Š” ์•ˆ์ „ํ•œ ๊ธฐ์ค€์ ์ด ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.


๐Ÿ™Œ ํ”„๋กœ์ ํŠธ GitHub ์ €์žฅ์†Œ