Is your feature request related to a problem? Please describe.
Right now if a constant is used by multiple consumers, we do not fold constant operations on each branch of a fork.
Describe the solution you'd like
We should add a pass that folds constants with multiple users by duplicating constants. This allows more aggressive fusion at the cost of model size.
Describe alternatives you've considered
We should allow users to optionally remove this pass if model size is an important metric.
Additional context
Is your feature request related to a problem? Please describe.
Right now if a constant is used by multiple consumers, we do not fold constant operations on each branch of a fork.
Describe the solution you'd like
We should add a pass that folds constants with multiple users by duplicating constants. This allows more aggressive fusion at the cost of model size.
Describe alternatives you've considered
We should allow users to optionally remove this pass if model size is an important metric.
Additional context