As a systemic problem, public health cannot be addressed without considering other policy dimensions. Hence, a holistic approach across public policy areas is necessary to incorporate Health-for-All values into decision-making. However, such multisectoral interventions require public budgets that are effectively mapped into public health outcomes and indicators of their wider determinants. This budget-tagging procedure is high-cost, given that it is often done manually by domain experts. In this paper, we propose Categorical Perplexity-based Uncertainty Quantification (CPUQ), a novel, cost-effective Large Language Models (LLMs) framework that can be leveraged by policymakers to build budget-to-indicator and indicator-to-indicator mappings. This model-agnostic method employs categorical-style prompts to generate interpretable Bernoulli and categorical distributions for edges in a Text-attributed Graph, which is associated with the descriptions of the budget items and indicators. The prompting strategy proposed provides a novel way to incorporate models' uncertainty within the final outputs, enhancing accuracy and safety, We find that the budget-to-indicator mapping predicted by the framework aligns effectively with expert annotations, while when prompted to infer indicator-to-indicator networks, CPUQ estimates more nuanced relationships compared to alternative LLMs-based methods. Through our work, we hope to provide valuable insights into the strengths and weaknesses of leveraging LLMs to support public health budget planning, with the aim of promoting the implementation of the Health-for-All agenda across diverse governments and institutions.