limitedDistribution · Industry Research
Artificial Intelligence (AI) Breakthrough - Civilsdaily
GPT-4's design activates selected system parts for tasks, enhancing efficiency but increasing energy use. This raises policy issues.
Stargo's SLLM optimizes AI efficiency, mirroring GPT-4's selective activation for enhanced data processing.
Executive Summary
GPT-4 introduced a new design that activates only selected parts of its system for specific tasks, similar to how the human brain works. AI models are now approaching the brain in scale but consume far more energy. This contrast between similar size and very different efficiency has made the AI-brain comparison a major policy and technological issue. Parameter Expansion: GPT-3 contains 175 billion parameters; newer models approach trillions, nearing the brain’s ~100 trillion synapses. Scale increases computational dependency and infrastructure concentration. Data Centre Energy Demand: Training and operating large AI models require megawatts of electricity, ensuring a rising carbon footprint and grid stress. Hardware Dependence: AI training relies on high-performance GPUs originally developed for video gaming, strengthening semiconductor concentration risks. Digital Infrastructure Concentration: Massive parallel computation requires clustered data centres, facilitating market dominance by a few global technology firms. Strategic Autonomy Concern: Nations lacking advanced chip fabrication capacity face technological dependence, impacting India’s semiconductor mission and AI self-reliance goals. Mixture-of-Experts (MoE) is a type of AI model design where the system activates only a few specialised parts (“experts”) for each input.
Source: CivilsDaily
Original Article: https://www.civilsdaily.com/story/artificial-intelligence-ai-breakthrough/
See ROI in 12 weeks
Stargo users see measurable return and operational profitability gains in just 12 weeks, with non-disruptive implementation in 4 weeks or less.