This repository was archived by the owner on Mar 15, 2024. It is now read-only.

This repository was archived by the owner on Mar 15, 2024. It is now read-only.

Gradient accumulation code #240

Open

opened

on Jan 26, 2024

Can you provide the gradient accumulation code? Since I only have 1 gpu for training, the max batch size that I can use is 256, and the final result can not match paper's result. So I'm thinking about doing gradient accumulation, thanks!!!

to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests