MIF_E31231623/android/wisata_app/AR_IMPLEMENTATION_GUIDE.md

AR Feature Documentation - Explore Lumajang AR

Overview

This document provides complete technical documentation for the Augmented Reality (AR) feature implementation in the "Explore Lumajang AR" Flutter tourism application.

Architecture

AR System Architecture

┌─────────────────────────────────────────┐
│      AR View Screen (UI Layer)          │
│  - Handles user interactions            │
│  - Displays AR visualization            │
│  - Gesture recognition                  │
└────────────┬────────────────────────────┘
             │
             ↓
┌─────────────────────────────────────────┐
│      AR Service (Business Logic)        │
│  - Scene state management               │
│  - Object transformations               │
│  - Gesture handling                     │
│  - Plane detection coordination         │
└────────────┬────────────────────────────┘
             │
             ↓
┌─────────────────────────────────────────┐
│      AR Models (Data Layer)             │
│  - ArScene: Scene state                 │
│  - ArObject: 3D objects                 │
│  - ArPlane: Detected surfaces           │
└────────────┬────────────────────────────┘
             │
             ↓
┌─────────────────────────────────────────┐
│   ar_flutter_plugin (AR Engine)         │
│  - ARCore integration                   │
│  - Camera control                       │
│  - Plane detection                      │
│  - Model rendering                      │
└─────────────────────────────────────────┘

Components

1. AR Service (ar_service.dart)

Purpose: Central service for managing AR scene state and operations

Key Responsibilities:

• Scene state management
• Object placement and manipulation
• Gesture recognition
• Plane detection management
• State notifications via ChangeNotifier

Main Methods:

• placeObject() - Place 3D model on detected plane
• handleRotation() - Rotate object via gesture
• handlePinch() - Scale object via pinch gesture
• handleDrag() - Move object via drag gesture
• simulateSurfaceDetection() - Simulate AR plane detection

2. AR Models

ArObject (ar_object.dart)

Represents a 3D object in AR space with transformation properties.

Properties:

• id: Unique identifier
• modelPath: Path to GLB/GLTF file
• position: 3D coordinates (Vector3)
• rotation: Rotation in radians (Vector3)
• scale: Scale factor (double)
• isVisible: Visibility state

Methods:

• updatePosition() - Update position
• updateRotation() - Update rotation
• updateScale() - Update scale
• rotate() - Apply rotation delta
• move() - Apply position offset
• reset() - Reset to initial state

ArPlane (ar_plane.dart)

Represents a detected AR plane/surface.

Properties:

• id: Plane identifier
• center: Center position (Vector3)
• normal: Surface normal (Vector3)
• extent: Plane dimensions (Vector2)
• type: Plane type (horizontal_up, horizontal_down, vertical)
• isTracked: Tracking status

Methods:

• containsPoint() - Check if point is on plane
• getRandomPointOnPlane() - Get random surface point

ArScene (ar_scene.dart)

Represents the complete AR scene state.

Properties:

• detectedPlanes: List of detected surfaces
• objects: List of placed objects
• selectedObject: Currently selected object
• isSurfaceDetectionActive: Detection status

Methods:

• addPlane() / removePlane() - Plane management
• addObject() / removeObject() - Object management
• selectObject() - Select object
• clearScene() - Clear all AR content

3. AR View Screen (ar_view_screen.dart)

Screens:

1. Scanning Screen - Shows while detecting surfaces
2. Placing Screen - Shows when objects are placed
3. Manipulation Screen - Allows gesture-based interaction

UI Elements:

• Top bar with destination info and detection status
• Center display (scanner animation or object visualization)
• Status bar showing plane and object counts
• Bottom controls for object placement and manipulation

Gesture Support:

• Pan: Rotate object (horizontal) or move vertically
• Pinch: Scale object
• Tap: Place object or select object

4. Utilities (ar_utils.dart)

ArUtils Class:

• Math operations (degrees ↔ radians conversion)
• 3D vector operations
• Ray-plane intersection
• Angle normalization
• Performance metrics tracking

ArPerformanceMetrics Class:

• Frame time tracking
• FPS calculation
• Performance monitoring

Data Flow

Object Placement Flow

User taps "Place Object"
       ↓
Check for detected planes
       ↓
Get random point on plane
       ↓
Create ArObject instance
       ↓
Add to ArScene
       ↓
Select object automatically
       ↓
Notify UI (ListenableBuilder)
       ↓
UI renders object

Gesture Handling Flow

User performs gesture (pan/pinch/drag)
       ↓
Gesture detector captures movement
       ↓
Calculate delta/scale
       ↓
Call ArService method
       ↓
Update selected object
       ↓
Notify listeners
       ↓
UI rebuilds with new transformation

Integration Guide

1. Update pubspec.yaml

dependencies:
  ar_flutter_plugin: ^0.7.3
  vector_math: ^2.1.4
  provider: ^6.4.0

Run flutter pub get

2. Android Configuration

Ensure AndroidManifest.xml includes:

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera.ar" android:required="true" />
<meta-data android:name="com.google.ar.core" android:value="required" />

3. Runtime Permissions

The app requires camera permission at runtime on Android 6+:

// Handled by ar_flutter_plugin
// Request camera permission before launching AR view

4. Navigation

From Detail Screen → AR Screen:

Navigator.pushNamed(
  context, 
  '/ar',
  arguments: destination,  // Pass destination object
)

Usage Examples

Initialize AR Service

final destination = DestinationService.byId('tumpak-sewu');
final arService = ArService(destination: destination);

Place an Object

arService.placeObject(
  objectId: 'waterfall_1',
  modelPath: 'assets/models/waterfall.glb',
  objectName: 'Tumpak Sewu Waterfall',
)

Rotate Object

arService.rotateObject(0.1); // Rotate by 0.1 radians

Scale Object

arService.zoomIn();   // Multiply scale by 1.2
arService.zoomOut();  // Divide scale by 1.2

Clear Scene

arService.clearScene(); // Remove all objects and planes

3D Model Integration

Supported Formats

• GLB (recommended, binary format)
• GLTF (with separate assets)

Model Requirements

• Polygon Count: 10,000-50,000 triangles
• File Size: Under 10-15 MB
• Textures: Compressed (WebP/ASTC preferred)
• Optimization: Mobile-optimized materials

Adding New Models

1. Create/download 3D model (GLB or GLTF)
2. Optimize for mobile performance
3. Place in assets/models/
4. Update destination_service.dart:

   arModelPath: 'assets/models/your_model.glb',
   

5. Update pubspec.yaml:

   assets:
     - assets/models/your_model.glb
   

Model Optimization Pipeline

Raw Model
    ↓
Cleanup (remove unnecessary geometry)
    ↓
Decimation (reduce polygon count)
    ↓
Texture Baking
    ↓
Export as GLB (compressed)
    ↓
Test in AR

Error Handling

ARCore Not Supported

When ARCore is not available:

1. Error message displayed to user
2. Fallback option to return to destination detail
3. No AR functionality attempted

Model Loading Failures

When model file is missing or corrupted:

1. Error logged to console
2. User notified via snackbar
3. UI remains responsive

Plane Detection Timeout

If plane not detected within timeout:

1. Continue surface detection
2. Show instruction message
3. Allow user to move device

Performance Optimization

For Mobile Devices

1. Model Optimization

   • Keep polygon count under 50k
   • Use compressed textures
   • Single material per model when possible

2. Scene Management

   • Limit to 1-2 placed objects per scene
   • Disable off-screen object rendering
   • Clean up removed objects immediately

3. Gesture Handling

   • Throttle gesture updates (16ms minimum)
   • Batch transformation updates
   • Avoid real-time physics calculations

4. Memory Management

   • Preload models on destination selection
   • Clear scene when exiting AR
   • Dispose services properly

Testing

Unit Tests

test('ArObject position update', () {
  final obj = ArObject(
    id: 'test',
    modelPath: 'path',
    name: 'Test',
  );
  
  final newPos = Vector3(1, 2, 3);
  obj.updatePosition(newPos);
  
  expect(obj.position, equals(newPos));
});

Integration Tests

testWidgets('AR view displays placing screen', (WidgetTester tester) async {
  await tester.pumpWidget(const ArViewScreen());
  expect(find.text('Ready to place object'), findsOneWidget);
});

Debugging

Enable AR Debug Mode

ArDebugInfo.getDebugInfo(
  planeCount: arService.planeCount,
  objectCount: arService.objectCount,
  isSurfaceDetecting: arService.isSurfaceDetecting,
);

Performance Monitoring

final metrics = ArPerformanceMetrics();
metrics.recordFrameTime(16); // 16ms frame time
print('FPS: ${metrics.getCurrentFps()}');

Logging

ArUtils.logArOperation('Place Object', 'Object placed at (1,2,3)');

Troubleshooting

Issue: Objects not appearing

Causes:

• Model path incorrect
• Model file missing from assets
• Plane not detected yet

Solution:

• Verify path in destination_service.dart
• Check assets/models/ directory
• Move device to detect surface

Issue: Performance drops

Causes:

• High polygon count
• Too many objects placed
• Large uncompressed textures

Solution:

• Optimize model file size
• Limit objects per scene
• Compress textures

Issue: Touch not responding

Causes:

• No object selected
• Gesture detector disabled
• Service not initialized

Solution:

• Tap to place/select object first
• Check ArService initialization
• Verify gesture listeners are active

Future Enhancements

• Support for multiple simultaneous objects
• Custom animation playback
• Object collision detection
• Physics-based interactions
• Cloud-based model streaming
• Multi-user AR collaboration
• Custom gesture patterns
• Model texture customization
• Lighting control
• Video capture and sharing

Resources

• AR Flutter Plugin Docs
• Vector Math Package
• ARCore Documentation
• GLB/GLTF Format
• Blender 3D Modeling
• Google Play Services Setup

Support

For issues or questions:

1. Check troubleshooting section
2. Review code comments
3. Check AR plugin documentation
4. File GitHub issue with debugging info