Fix llava-v1.5-7b contrib: remove docstring wrapping entire module#113
Open
jimburtoft wants to merge 1 commit intoaws-neuron:mainfrom
Open
Fix llava-v1.5-7b contrib: remove docstring wrapping entire module#113jimburtoft wants to merge 1 commit intoaws-neuron:mainfrom
jimburtoft wants to merge 1 commit intoaws-neuron:mainfrom
Conversation
Hoomaaan
approved these changes
Apr 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This is a fix to an existing contrib model, not a new contribution.
The entire
modeling_llava.pyfile is wrapped in an extraneous"""on line 17, turning all the custom class definitions into a docstring. This causes aSyntaxErroron import -- none of the custom classes (NeuronLlavaAttention,NeuronLlavaDecoderLayer,NeuronLlavaForCausalLM, etc.) are actually defined.The model previously appeared to work only because
NeuronLlavaForCausalLMinherits fromNeuronBaseForCausalLMvia the base class, and the__init__.pyimports fell back to defaults. But none of the llava-specific customizations (attention, MLP, decoder layer overrides) were active.Changes
Removed the extraneous
"""on line 17 ofmodeling_llava.pyso the module parses correctly and all custom classes are properly defined.Validation Results
Files Changed
contrib/models/llava-v1.5-7b/src/modeling_llava.py"""that wrapped entire moduleTesting
Compiled and ran on inf2.8xlarge with TP=2, Neuron SDK 2.27, BF16. Model compiles, loads, and generates coherent text. Before the fix,
import modeling_llavaraisedSyntaxError.