Objective: Recent quantification research on Parkinson's disease (PD) integrates wearable technology with machine learning methods, indicating a strong potential for practical applications. However, the effectiveness of these techniques is influenced by environmental settings and is hardly applied in real-world situations. This paper aims to propose an effective feature assessment framework to automatically rate the severity of PD motor symptoms from short-term motor tasks, and then classify different PD severity levels in the real world. Methods: This paper identified specific PD motor symptoms using a novel feature-assessment framework at both segment-level and sample-level. Features were selected after calculating SHapley Additive exPlanation(SHAP) value, and verified by different machine learning methods with appropriate parameters. This framework has been verified on real-world data from 100 PD patients performing Unified Parkinson's Disease Rating Scale(UPDRS)-recommended short motor tasks, each task lasting 20-50 seconds. Results: The sensitivity for recognizing motor fluctuations reached 88% in tremor recognition. Additionally, LightGBM achieved the highest accuracy for early detection(92.59%) and achieved 71.58% in fine-grained severity classification using 31 selected features. Conclusion: This paper reports the first effort to assess multi-level and multi-scale features for automatic quantification of motor symptoms and PD severity levels. The proposed framework has been proven effective in assessing key PD information for recognition during short-term tasks. Significance: The explanatory analysis of digital features in this study provides more prior knowledge for PD self-assessment in a free-living environment.