Repository navigation
Feedback about NLP From Scratch: Generating Names with a Character-Level RNN #3791
Description
There is the following issue on this page: https://docs.pytorch.org/tutorials/intermediate/char_rnn_generation_tutorial.html
I might be missing something, but in the code on this tutorial, there is no nonlinearity in the recurring part of the RNN, degrading all of the internal updates to a single linear function.
Activity
pranay-3108 commentedon Mar 14, 2026
Hi, I tried running the tutorial locally to understand the issue better
The code runs correctly on my side and the model trains and generates names as expected. While looking through the RNN code I also noticed that the hidden state update is done with a linear layer:
hidden = self.i2h(input_combined)So there is no explicit non-linearity applied to the hidden state update I’m not sure if this was intentional to keep the example simple for learning purposes, or if adding something like
torch.tanhwould make the recurrence closer to a typical RNN cellJust sharing my observation after running the example If I misunderstood something, please to correct me