Updated 2mo ago
by EmbrasureAI
spark-observability-skills
Open-source agent skills for debugging and optimizing Apache Spark workloads
54 GitHub stars (54)
+2 in 16 days
| Period | New stars |
|---|---|
| Oct 1 | 0 |
| Oct 2 | +1 |
| Oct 3 | +1 |
| Oct 4 | 0 |
| Oct 5 | 0 |
| Oct 6 | 0 |
| Oct 7 | 0 |
| Oct 8 | 0 |
| Oct 9 | 0 |
| Oct 10 | 0 |
| Oct 11 | 0 |
| Oct 12 | 0 |
| Oct 13 | 0 |
| Oct 14 | 0 |
| Oct 15 | 0 |
| Oct 16 | 0 |
- 3 skills (3 skills found in its SKILL.md files)
- 14 forks (14 forks)
- 2mo ago last update (Last updated 2mo ago)
Install
Skills CLI
npx skills add EmbrasureAI/spark-observability-skills
Asks which skills and agents to install. Just one: npx skills add EmbrasureAI/spark-observability-skills --skill debug-slow-spark-job — add -g for every project.
Claude Code
git clone https://github.com/EmbrasureAI/spark-observability-skillscp -r "spark-observability-skills/skills/debug-slow-spark-job" ~/.claude/skills/
Copies the debug-slow-spark-job skill into ~/.claude/skills/ for every project (use .claude/skills/ for one project). Repeat for the other skills listed below.
Plugin
/plugin marketplace add EmbrasureAI/spark-observability-skills
Run inside Claude Code if the repository ships a plugin marketplace, then pick plugins from /plugin.
Skills can run code on your machine — read the SKILL.md first.
What it does
In the author's own words, from each skill's SKILL.md.
-
debug-slow-spark-job
Diagnose slow, expensive, or regressed Apache Spark and PySpark applications by comparing runtime evidence against a healthy run. Use for long stages, stragglers, skew, shuffle, spill, garbage collection, poor parallelism, small files, slow scans, scheduler delay, executor imbalance, and unexplained compute-cost growth.
skills/debug-slow-spark-job/SKILL.md on GitHub (opens in a new tab) -
debug-spark-failure
Diagnose failed Apache Spark and PySpark applications from History Server evidence, logs, and cluster-manager state. Use for driver or executor crashes, out-of-memory errors, fetch failures, task exceptions, timeouts, repeated retries, aborted stages, and intermittent production failures.
skills/debug-spark-failure/SKILL.md on GitHub (opens in a new tab) -
optimize-spark-sql-plan
Optimize Apache Spark SQL and DataFrame queries using the final Adaptive Query Execution plan and runtime statistics rather than source code alone. Use to reduce runtime, shuffle, spill, scan cost, skew, join amplification, Python UDF overhead, poor partitioning, or unnecessary work while preserving query semantics.
skills/optimize-spark-sql-plan/SKILL.md on GitHub (opens in a new tab)
No skill matches that filter.
Review before you install
A skill is a set of instructions and scripts your agent will follow, and it can run code on your machine. Read the SKILL.md and any scripts first — this listing is not a security review.