Skip to content

Commit 0947191

Browse files
vfdev-5vfdev-5
authored
Fixed iteration visual value in fbresearch_logger (#3459)
Description: - Fixed iteration visual value in fbresearch_logger Before this PR (10 -> 9 -> 8 -> 7 but should be always 10): ``` Training: start epoch [1/15] Epoch [1/15] [10/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [1/15] [20/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Training: Epoch [1/15] Total time: 0:00:00 (0.0000 s / it) Training: start epoch [2/15] Epoch [2/15] [9/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [2/15] [19/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Training: Epoch [2/15] Total time: 0:00:00 (0.0000 s / it) Training: start epoch [3/15] Epoch [3/15] [8/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [3/15] [18/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Training: Epoch [3/15] Total time: 0:00:00 (0.0000 s / it) Training: start epoch [4/15] Epoch [4/15] [7/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [4/15] [17/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Training: Epoch [4/15] Total time: 0:00:00 (0.0000 s / it) Training: start epoch [5/15] Epoch [5/15] [6/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [5/15] [16/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s ``` with the PR: ``` Training: start epoch [1/15] Epoch [1/15] [10/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [1/15] [20/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Training: Epoch [1/15] Total time: 0:00:00 (0.0000 s / it) Training: start epoch [2/15] Epoch [2/15] [10/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [2/15] [20/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Training: Epoch [2/15] Total time: 0:00:00 (0.0000 s / it) Training: start epoch [3/15] Epoch [3/15] [10/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [3/15] [20/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Training: Epoch [3/15] Total time: 0:00:00 (0.0000 s / it) Training: start epoch [4/15] Epoch [4/15] [10/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [4/15] [20/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Training: Epoch [4/15] Total time: 0:00:00 (0.0000 s / it) Training: start epoch [5/15] Epoch [5/15] [10/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s Epoch [5/15] [20/20]: ETA: 0:00:00 42.0000 Iter time: 0.0000 s Data prep time: 0.0000 s ``` Check list: - [x] New tests are added (if a new feature is added) - [ ] New doc strings: description and/or example code are in RST format - [ ] Documentation is updated (if required)
1 parent 63550c6 commit 0947191

File tree

2 files changed

+26
-1
lines changed

2 files changed

+26
-1
lines changed

‎ignite/handlers/fbresearch_logger.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,7 @@ def log_every(self, engine: Engine, optimizer: Optional[torch.optim.Optimizer] =
154154
if torch.cuda.is_available():
155155
cuda_max_mem = f"GPU Max Mem: {torch.cuda.max_memory_allocated() / MB:.0f} MB"
156156

157-
current_iter = engine.state.iteration % (engine.state.epoch_length + 1)
157+
current_iter = ((engine.state.iteration - 1) % engine.state.epoch_length) + 1
158158
iter_avg_time = self.iter_timer.value()
159159

160160
eta_seconds = iter_avg_time * (engine.state.epoch_length - current_iter)

‎tests/ignite/handlers/test_fbresearch_logger.py‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,3 +104,28 @@ def test_fbrlogger_with_state_attrs(mock_logger):
104104
trainer.run(data=[10], epoch_length=1, max_epochs=1)
105105
attrs = "alpha: 3.8990 beta: 12.2100 gamma: [21.0000, 6.0000]"
106106
assert attrs in fbr.logger.info.call_args_list[-2].args[0]
107+
108+
109+
def test_fbrlogger_iters_values_bug(mock_logger):
110+
max_epochs = 15
111+
every = 10
112+
data_size = 20
113+
trainer = Engine(lambda e, b: 42)
114+
fbr = FBResearchLogger(logger=mock_logger, show_output=True)
115+
fbr.attach(trainer, "Training", every=every)
116+
trainer.run(data=range(data_size), max_epochs=max_epochs)
117+
118+
expected_epoch = 1
119+
expected_iters = [i for i in range(every, data_size + 1, every)]
120+
n_calls_per_epoch = data_size // every
121+
i = 0
122+
for call_args in fbr.logger.info.call_args_list:
123+
msg = call_args.args[0]
124+
if msg.startswith("Epoch"):
125+
expected_iter = expected_iters[i]
126+
assert f"Epoch [{expected_epoch}/{max_epochs}] [{expected_iter}/{data_size}]" in msg
127+
if i == n_calls_per_epoch - 1:
128+
expected_epoch += 1
129+
i += 1
130+
if i == n_calls_per_epoch:
131+
i = 0

0 commit comments

Comments
 (0)