fix(ignis): retain Echo recovery warnings during shutdown

Report failed reward saves independently of the tick callback with owner and receipt identifiers. Verify the approved lightweight consumption, inert-copy and failure/retry contract using native inventories and real persistence. Keep cross-save hard-crash recovery an explicit operator limitation; add no autosave-length restrictions or transaction protocol.

README.md

@@ -37,7 +37,7 @@ Take uncancelled void damage, ascend while gliding with elytra in the same world
 
 The earned Echo Shard is named **Echo of the Abyss**, with cryptic lore and a UUID-bound receipt. Right-click it to activate permanent obfuscated-name advancement announcements. It does not hide general identity or alter advancement titles/descriptions. Only the bound owner may use it, and activation cannot grant twice. Lost, unconsumed rewards can be earned again by repeating the feat. Admins can also grant a bound Echo.
 
-Full inventories retain a pending reward; online delivery retries every five seconds using the same receipt identity. Temporary inventory/drop/swap locks prevent ordinary moves during the short activation commit. Disk failures do not consume the item on activation; failed completion saves produce a player-facing recovery message rather than pretending the reward was saved.
+Full inventories retain a pending reward; online delivery retries every five seconds using the same receipt identity. Temporary inventory/drop/swap locks prevent ordinary moves during the short activation commit. Disk failures do not consume the item on activation, so it can be retried. Failed challenge-completion saves log the owner UUID and receipt ID for administrator recovery even if shutdown prevents the player callback; an online player is also notified when scheduling remains available. Pending delivery acknowledgements can retry without offering another copy while the original remains in the inventory.
 
 Announcement handling uses Purpur's Adventure `PlayerAdvancementDoneEvent.message`, replacing only the first name argument of the three vanilla advancement translations. It preserves silent events and unrelated/custom message formats. An advancement-only ProtocolLib output backstop also covers Purpur's affected-player-only branch, which sends the original pre-event component rather than the event's replacement. The backstop identifies an unambiguous player UUID in the name slot's entity hover and checks saved activation; it does not infer identity from name text, touch titles/descriptions or broaden recipients. Ordinary/custom chat remains unchanged. Native activation/reopen, event/delivery branches, packet codecs and plugin lifecycle tests cover this fix; the exact live report setting/format remains unconfirmed. Existing Stealth outbound masking sees the obfuscated alias rather than the original name. Live coexistence remains unverified.
 
@@ -116,12 +116,14 @@ Offline targets use UUIDs already present in Ignis state; online exact names are
 
 Writes use a same-directory temporary file, flush it to disk, and require atomic replacement. Unsupported atomic moves or write failures retain the prior active snapshot and fail the mutation; there is no non-atomic fallback. Abandoned temporary files are ignored, never auto-promoted. This does not guarantee survival of arbitrary storage hardware or power failures.
 
+For a reported failed Echo award, restore writable storage first and match the logged owner UUID/receipt to the player. Inspect saved activation and pending rewards before granting anything: pending rewards retry automatically when inventory space is available. If the failed award needs replacement, use the existing operator-only `/ignisadmin give <target> echo` after review. A UUID not yet saved in Ignis must reconnect so its exact online name can be targeted. Do not reset earned progression merely to remove an inert leftover item, and do not grant a second reward for an already activated benefit.
+
 If loading fails, preserve the original file for investigation. Under an approved maintenance plan, stop the server, back up current data, restore a verified matching-version backup, and restart. Do not delete state to recover: that loses earned progress and invalidation records. Do not hand-edit the binary file or import another plugin's state. Restoring an old backup can also restore older revocation/progress state; review that consequence before doing it.
 
 ## Verification limits
 
 JUnit tests exercise real temporary-file persistence, serialized writes, fault injection, threshold policies, receipt/owner/reset rules, challenge transitions, announcement components, command execution/completion, mocked Bukkit boundaries, metadata, and static workflow guardrails. YAML/shell parsing is not remote CI verification.
 
-Live Purpur startup/shutdown, event sequencing, item appearance/consumption, inventory durability across hard crashes, protections, and Stealth/client compatibility still require a manual smoke test. Minecraft inventory persistence and this plugin's state file are not a single atomic transaction: a hard crash between item delivery and Minecraft saving inventory needs operator recovery review. If disconnect/death happens during activation, the durable benefit can commit before the physical item is removed; any retained copy is inert, not another usable unlock. These limitations prevent claiming complete live/crash acceptance or deploying automatically.
+Live Purpur startup/shutdown, event sequencing, item appearance/consumption, inventory durability across hard crashes, protections, and Stealth/client compatibility still require a manual smoke test. Minecraft inventory persistence and this plugin's state file are not a single atomic transaction: a hard crash between item delivery and Minecraft saving inventory needs operator recovery review. If disconnect/death happens during activation, the durable benefit can commit before the physical item is removed; any retained copy is inert, not another usable unlock. The approved lightweight contract guarantees one-item consumption in uninterrupted play and one permanent owner activation, not crash-atomic physical consumption. Hard crashes between the two saves may require administrator recovery; extra copies left by interruptions cannot activate again. No cross-save transaction protocol, forced player saves or autosave-length item restrictions are added. This is not a claim of live verification or permission to deploy.
 
 Before a later approved production release, test: void damage/elytra/End Stone success and all interruption paths; other players and lookalikes failing activation; full inventory and failed saves; logout/death during activation; reset/re-earn invalidation; all advancement types and hover/click content with Stealth; admin permission/confirmation/offline UUID cases; direct-drink attribution after reconnect/restart and external effect replacement; skin outages; manual/shift/Crafter ingredient protection; tradable Mask issuer/holder resets; own/other-player helmet rendering, inventory interactions, and preservation of external invisibility; real consumed-pearl launch/statistic ordering; Dagger regular/shift/Crafter recipes, stale receipts, vault-key protection, target visibility, thin obstacles, scaled/posed bodies, hazardous/unsupported exits, world/chunk boundaries, cooldowns across copies/disconnect, and cancelled or redirected protection-plugin teleports.

src/main/java/games/dmg/spigotignis/EchoGameplay.java

@@ -60,13 +60,19 @@ public final class EchoGameplay implements Listener {
             UUID owner = player.getUniqueId(), receipt = UUID.randomUUID();
             long generation = states.snapshot().players().getOrDefault(owner, PlayerState.empty()).track(Track.VOID).generation();
             states.update(state -> state.players().getOrDefault(owner, PlayerState.empty()).track(Track.VOID).generation() == generation
-                    ? EchoRewards.award(state, owner, receipt) : state).whenComplete((state, error) -> onServer(() -> {
+                    ? EchoRewards.award(state, owner, receipt) : state).whenComplete((state, error) -> {
+                        // Report independently of the tick callback: disable can prevent its delivery.
                         if (error != null) {
-                            plugin.getLogger().warning("Echo reward save failed; administrator recovery is required.");
-                            Player online = plugin.getServer().getPlayer(owner);
-                            if (online != null) online.sendMessage("Your void reward could not be saved. Please contact an administrator.");
-                        } else retryDeliveries();
-                    }));
+                            plugin.getLogger().warning("Echo reward save failed for " + owner + " (receipt " + receipt
+                                    + "); administrator recovery is required.");
+                        }
+                        onServer(() -> {
+                            if (error != null) {
+                                Player online = plugin.getServer().getPlayer(owner);
+                                if (online != null) online.sendMessage("Your void reward could not be saved. Please contact an administrator.");
+                            } else retryDeliveries();
+                        });
+                    });
         }
     }
 

src/nativeTest/java/games/dmg/spigotignis/NativeEchoLifecycleTest.java

@@ -0,0 +1,229 @@
+package games.dmg.spigotignis;
+
+import static org.junit.jupiter.api.Assertions.*;
+import static org.mockito.Mockito.*;
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.UUID;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import net.minecraft.server.level.ServerPlayer;
+import net.minecraft.world.entity.EntityEquipment;
+import net.minecraft.world.entity.player.Inventory;
+import org.bukkit.Bukkit;
+import org.bukkit.craftbukkit.CraftServer;
+import org.bukkit.craftbukkit.entity.CraftPlayer;
+import org.bukkit.craftbukkit.inventory.CraftInventoryPlayer;
+import org.bukkit.craftbukkit.scheduler.CraftScheduler;
+import org.bukkit.event.block.Action;
+import org.bukkit.event.player.PlayerInteractEvent;
+import org.bukkit.inventory.EquipmentSlot;
+import org.bukkit.plugin.java.JavaPlugin;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
+
+class NativeEchoLifecycleTest {
+    @TempDir Path directory;
+    @BeforeAll static void bootstrap() throws Exception { NativeRuntime.bootstrap(); }
+
+    @ParameterizedTest
+    @CsvSource({"0,1", "0,3", "40,1", "40,3"})
+    void actualInventoryConsumesExactlyOneOnceAndReopensActivated(int slot, int amount) throws Exception {
+        try (var f = new Fixture(directory)) {
+            var echo = f.items.create(f.token);
+            echo.setAmount(amount);
+            f.inventory.setItem(slot, echo);
+            var use = f.use(slot);
+            f.gameplay.onUse(use);
+            f.gameplay.onUse(use);
+            f.callback();
+            assertTrue(f.gameplay.activated(f.owner));
+            assertEquals(amount - 1, f.count());
+            f.gameplay.onUse(use);
+            assertEquals(amount - 1, f.count());
+            assertFalse(EchoRewards.valid(f.states.snapshot(), f.owner, f.token));
+            try (var reopened = StateService.open(f.disk).get(5, TimeUnit.SECONDS)) {
+                assertTrue(new EchoGameplay(f.plugin, reopened, f.items).activated(f.owner));
+                assertFalse(EchoRewards.valid(reopened.snapshot(), f.owner, f.token));
+            }
+        }
+    }
+
+    @ParameterizedTest
+    @org.junit.jupiter.params.provider.ValueSource(strings = {"disconnect", "death", "disable"})
+    void interruptedCopiesRemainInertAfterReopen(String interruption) throws Exception {
+        try (var f = new Fixture(directory)) {
+            var copy = f.items.create(f.token);
+            f.inventory.setItem(0, copy);
+            f.gameplay.onUse(f.use(0));
+            var callback = f.awaitCallback();
+            assertTrue(f.gameplay.activated(f.owner));
+            if (interruption.equals("disconnect")) {
+                var quit = mock(org.bukkit.event.player.PlayerQuitEvent.class);
+                when(quit.getPlayer()).thenReturn(f.player);
+                f.gameplay.onQuit(quit);
+                when(f.server.getPlayer(f.owner)).thenReturn(null);
+                callback.run();
+                assertEquals(1, f.count());
+            } else if (interruption.equals("death")) {
+                // The world/drop boundary removes the item before the pending callback.
+                f.inventory.clear();
+                var death = mock(org.bukkit.event.entity.PlayerDeathEvent.class);
+                when(death.getEntity()).thenReturn(f.player);
+                f.gameplay.onDeath(death);
+                callback.run();
+                f.inventory.setItem(0, copy); // later pickup of the retained drop
+            } else {
+                f.enabled.set(false); // Bukkit cancels the queued callback on disable; do not execute it.
+                assertEquals(1, f.count());
+            }
+            try (var reopened = StateService.open(f.disk).get(5, TimeUnit.SECONDS)) {
+                var resumed = new EchoGameplay(f.plugin, reopened, f.items);
+                f.enabled.set(true);
+                when(f.server.getPlayer(f.owner)).thenReturn(f.player);
+                assertTrue(resumed.activated(f.owner));
+                resumed.onUse(f.use(0));
+                assertEquals(1, f.count(), "interrupted copies are inert, not consumed a second time");
+                assertFalse(EchoRewards.valid(reopened.snapshot(), f.owner, f.token));
+                assertTrue(f.callbacks.isEmpty(), "a spent receipt cannot schedule another activation");
+            }
+        }
+    }
+
+    @ParameterizedTest
+    @org.junit.jupiter.params.provider.ValueSource(strings = {"foreign", "unknown", "reset", "denied"})
+    void onlyTheCurrentOwnerReceiptCanActivateAndProtectionIsRespected(String rejection) throws Exception {
+        try (var f = new Fixture(directory)) {
+            var token = rejection.equals("unknown") ? new EchoToken(f.owner, UUID.randomUUID(), 0) : f.token;
+            var echo = f.items.create(token);
+            f.inventory.setItem(0, echo);
+            if (rejection.equals("foreign")) { when(f.player.getUniqueId()).thenReturn(UUID.randomUUID()); }
+            if (rejection.equals("reset")) {
+                f.states.update(state -> ProgressAdministration.reset(state, f.owner, java.util.Set.of(Track.VOID))).get(5, TimeUnit.SECONDS);
+                f.states.update(state -> EchoRewards.award(state, f.owner, UUID.randomUUID())).get(5, TimeUnit.SECONDS);
+            }
+            var use = f.use(0);
+            if (rejection.equals("denied")) { use.setUseItemInHand(org.bukkit.event.Event.Result.DENY); }
+            f.gameplay.onUse(use);
+            assertFalse(f.gameplay.activated(f.owner));
+            assertEquals(echo, f.inventory.getItem(0));
+            assertTrue(f.callbacks.isEmpty());
+        }
+    }
+
+    @org.junit.jupiter.api.Test
+    void failedActivationLeavesRealItemsUnconsumedAndCanBeRetried() throws Exception {
+        try (var f = new Fixture(directory)) {
+            var echo = f.items.create(f.token);
+            echo.setAmount(3);
+            f.inventory.setItem(0, echo);
+            f.failSave.set(true);
+            f.gameplay.onUse(f.use(0));
+            f.callback();
+            assertFalse(f.gameplay.activated(f.owner));
+            assertEquals(3, f.count());
+            verify(f.player).sendMessage(contains("It has not been consumed; try again"));
+            f.failSave.set(false);
+            f.gameplay.onUse(f.use(0));
+            f.callback();
+            assertTrue(f.gameplay.activated(f.owner));
+            assertEquals(2, f.count());
+        }
+    }
+
+    @org.junit.jupiter.api.Test
+    void actualFullInventoryDefersAndFailedAcknowledgementRetriesWithoutAnotherOffer() throws Exception {
+        try (var f = new Fixture(directory)) {
+            for (int slot = 0; slot < 36; slot++) {
+                f.inventory.setItem(slot, new org.bukkit.inventory.ItemStack(org.bukkit.Material.COBBLESTONE, 64));
+            }
+            f.gameplay.retryDeliveries();
+            assertEquals(0, f.count());
+            assertTrue(f.states.snapshot().players().get(f.owner).pendingRewards().containsKey(f.token.receipt()));
+            assertTrue(f.callbacks.isEmpty());
+            f.inventory.setItem(5, null);
+            f.failSave.set(true);
+            f.gameplay.retryDeliveries();
+            f.gameplay.retryDeliveries();
+            f.callback();
+            assertEquals(1, f.count());
+            assertTrue(f.states.snapshot().players().get(f.owner).pendingRewards().containsKey(f.token.receipt()));
+            f.failSave.set(false);
+            f.gameplay.retryDeliveries();
+            f.callback();
+            f.gameplay.retryDeliveries();
+            assertEquals(1, f.count());
+            assertTrue(f.states.snapshot().players().get(f.owner).pendingRewards().isEmpty());
+            try (var reopened = StateService.open(f.disk).get(5, TimeUnit.SECONDS)) {
+                assertTrue(reopened.snapshot().players().get(f.owner).pendingRewards().isEmpty());
+                assertTrue(EchoRewards.valid(reopened.snapshot(), f.owner, f.token));
+            }
+        }
+    }
+
+    private static final class Fixture implements AutoCloseable {
+        final UUID owner = UUID.randomUUID();
+        final EchoToken token = new EchoToken(owner, UUID.randomUUID(), 0);
+        final EchoItems items = new EchoItems();
+        final CraftServer server = mock(CraftServer.class);
+        final org.mockito.MockedStatic<Bukkit> platform;
+        final JavaPlugin plugin = mock(JavaPlugin.class);
+        final CraftPlayer player = mock(CraftPlayer.class);
+        final CraftInventoryPlayer inventory;
+        final LinkedBlockingQueue<Runnable> callbacks = new LinkedBlockingQueue<>();
+        final AtomicBoolean enabled = new AtomicBoolean(true), failSave = new AtomicBoolean();
+        final FileStateRepository disk;
+        final StateService states;
+        final EchoGameplay gameplay;
+        Fixture(Path directory) throws Exception {
+            platform = mockStatic(Bukkit.class, call -> switch (call.getMethod().getName()) {
+                case "getServer" -> server;
+                case "isPrimaryThread" -> true;
+                default -> call.callRealMethod();
+            });
+            var nativePlayer = mock(ServerPlayer.class);
+            when(nativePlayer.getBukkitEntity()).thenReturn(player);
+            when(player.getHandle()).thenReturn(nativePlayer);
+            inventory = new CraftInventoryPlayer(new Inventory(nativePlayer, new EntityEquipment()));
+            when(player.getInventory()).thenReturn(inventory);
+            when(player.getUniqueId()).thenReturn(owner);
+            when(plugin.getServer()).thenReturn(server);
+            when(plugin.getLogger()).thenReturn(java.util.logging.Logger.getAnonymousLogger());
+            when(plugin.isEnabled()).thenAnswer(call -> enabled.get());
+            when(server.getPlayer(owner)).thenReturn(player);
+            doReturn(List.of(player)).when(server).getOnlinePlayers();
+            var scheduler = mock(CraftScheduler.class);
+            when(server.getScheduler()).thenReturn(scheduler);
+            when(scheduler.runTask(eq(plugin), any(Runnable.class))).thenAnswer(call -> { callbacks.add(call.getArgument(1)); return null; });
+            disk = new FileStateRepository(directory.resolve("state.bin"));
+            states = StateService.open(new StateRepository() {
+                @Override public IgnisState load() throws IOException { return disk.load(); }
+                @Override public void save(IgnisState state) throws IOException {
+                    if (failSave.get()) { throw new IOException("simulated unavailable storage"); }
+                    disk.save(state);
+                }
+            }).get(5, TimeUnit.SECONDS);
+            states.update(state -> EchoRewards.award(state, owner, token.receipt())).get(5, TimeUnit.SECONDS);
+            gameplay = new EchoGameplay(plugin, states, items);
+        }
+        PlayerInteractEvent use(int slot) {
+            return new PlayerInteractEvent(player, Action.RIGHT_CLICK_AIR, inventory.getItem(slot), null,
+                    org.bukkit.block.BlockFace.SELF, slot == 40 ? EquipmentSlot.OFF_HAND : EquipmentSlot.HAND);
+        }
+        Runnable awaitCallback() throws InterruptedException {
+            var callback = callbacks.poll(5, TimeUnit.SECONDS);
+            assertNotNull(callback);
+            return callback;
+        }
+        void callback() throws InterruptedException { awaitCallback().run(); }
+        int count() {
+            return java.util.Arrays.stream(inventory.getContents()).filter(item -> items.identify(item).filter(token::equals).isPresent())
+                    .mapToInt(org.bukkit.inventory.ItemStack::getAmount).sum();
+        }
+        @Override public void close() { states.close(); platform.close(); }
+    }
+}

src/test/java/games/dmg/spigotignis/EchoRewardFailureTest.java

@@ -0,0 +1,92 @@
+package games.dmg.spigotignis;
+
+import static org.junit.jupiter.api.Assertions.*;
+import static org.mockito.Mockito.*;
+import java.io.IOException;
+import java.util.UUID;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import org.bukkit.Location;
+import org.bukkit.Material;
+import org.bukkit.Server;
+import org.bukkit.World;
+import org.bukkit.block.Block;
+import org.bukkit.entity.Player;
+import org.bukkit.event.entity.EntityDamageEvent;
+import org.bukkit.event.player.PlayerMoveEvent;
+import org.bukkit.plugin.java.JavaPlugin;
+
+class EchoRewardFailureTest {
+    @org.junit.jupiter.params.ParameterizedTest
+    @org.junit.jupiter.params.provider.ValueSource(strings = {"online", "offline", "disabled", "scheduler-rejected"})
+    void failedEarnedRewardReportsRecoveryEvenWithoutPlayerCallback(String mode) throws Exception {
+        var entered = new CountDownLatch(1);
+        var release = new CountDownLatch(1);
+        var repository = new StateRepository() {
+            @Override public IgnisState load() { return IgnisState.empty(); }
+            @Override public void save(IgnisState state) throws IOException {
+                entered.countDown();
+                try { if (!release.await(5, TimeUnit.SECONDS)) { throw new IOException("test barrier timeout"); } }
+                catch (InterruptedException failure) { Thread.currentThread().interrupt(); throw new IOException(failure); }
+                throw new IOException("simulated unavailable storage");
+            }
+        };
+        var enabled = new AtomicBoolean(true);
+        var plugin = mock(JavaPlugin.class);
+        when(plugin.isEnabled()).thenAnswer(call -> enabled.get());
+        var server = mock(Server.class);
+        when(plugin.getServer()).thenReturn(server);
+        var logger = mock(java.util.logging.Logger.class);
+        when(plugin.getLogger()).thenReturn(logger);
+        var player = mock(Player.class);
+        UUID owner = UUID.randomUUID();
+        when(player.getUniqueId()).thenReturn(owner);
+        if (mode.equals("online")) { when(server.getPlayer(owner)).thenReturn(player); }
+        var callbacks = new java.util.concurrent.LinkedBlockingQueue<Runnable>();
+        var scheduler = mock(org.bukkit.scheduler.BukkitScheduler.class);
+        when(server.getScheduler()).thenReturn(scheduler);
+        when(scheduler.runTask(eq(plugin), any(Runnable.class))).thenAnswer(call -> {
+            if (mode.equals("scheduler-rejected")) { throw new org.bukkit.plugin.IllegalPluginAccessException("simulated shutdown"); }
+            callbacks.add(call.getArgument(1));
+            return null;
+        });
+        var world = mock(World.class);
+        when(world.getUID()).thenReturn(UUID.randomUUID());
+        when(player.getWorld()).thenReturn(world);
+        var block = mock(Block.class);
+        when(block.getType()).thenReturn(Material.END_STONE);
+        when(world.getBlockAt(any(Location.class))).thenReturn(block);
+        try (var states = StateService.open(repository).get(5, TimeUnit.SECONDS)) {
+            var gameplay = new EchoGameplay(plugin, states, new EchoItems());
+            var damage = mock(EntityDamageEvent.class);
+            when(damage.getEntity()).thenReturn(player);
+            when(damage.getCause()).thenReturn(EntityDamageEvent.DamageCause.VOID);
+            when(damage.getFinalDamage()).thenReturn(4.0);
+            gameplay.onDamage(damage);
+            var move = mock(PlayerMoveEvent.class);
+            when(move.getPlayer()).thenReturn(player);
+            when(move.hasChangedPosition()).thenReturn(true);
+            when(move.getTo()).thenReturn(new Location(world, 0, 10, 0));
+            when(player.isGliding()).thenReturn(true);
+            gameplay.onMove(move);
+            when(player.isOnGround()).thenReturn(true);
+            gameplay.onMove(move);
+            assertTrue(entered.await(5, TimeUnit.SECONDS));
+            enabled.set(!mode.equals("disabled"));
+            release.countDown();
+            states.update(state -> state).handle((state, failure) -> null).get(5, TimeUnit.SECONDS);
+            verify(logger).warning(contains("administrator recovery"));
+            verify(logger).warning(contains(owner.toString()));
+            assertFalse(states.snapshot().players().containsKey(owner));
+            verify(server, never()).getPlayer(any(UUID.class));
+            if (mode.equals("online") || mode.equals("offline")) {
+                var callback = callbacks.poll(5, TimeUnit.SECONDS);
+                assertNotNull(callback);
+                callback.run();
+            } else { assertTrue(callbacks.isEmpty()); }
+            if (mode.equals("online")) { verify(player).sendMessage(contains("contact an administrator")); }
+            else { verify(player, never()).sendMessage(anyString()); }
+        } finally { release.countDown(); }
+    }
+}