DataHub

Overview

blue box with battery inside a hub with lines sticking out of a circle

What is DataHub?

DataHub is a distribution of open-source JupyterHub service that provides interactive computing environments to educators and students across campus using open source tools in the Jupyter ecosystem and beyond. The project has allowed Berkeley’s data science program to deploy scalable Jupyter infrastructure utilizing cloud computing resources. In addition, it enables users to interact remotely with a standardized and common computing environment through any web browser. Compared to local environments that run Jupyter, a cloud-based JupyterHub provides many conveniences, including pre-installed software, quicker access to course content, and computing flexibility that enables even users on Chromebooks or iPads to run Jupyter notebook.

Go to DataHub

Instructors Getting Started

Are you an educator interested in teaching data-driven/quantitative courses that require students to do computing? DataHub offers a streamlined workflow for your teaching needs. If you are interested to learn more about the tool, Schedule consultation with Service Lead from Learning Environment and Tools (LET) team.

Get yourself onboarded with DataHub in just three easy steps:

Step 1 - Create a GitHub repository

Step 2 - Install DataHub Link Generator plugin

  • The DataHub link generator is a Berkeley-specific version of the nbgitpuller link generator browser extension. Nbgitpuller is a tool that streamlines the assignment distribution process in course settings. When students click the unique links generated by this tool, they'll be taken directly to Jupyter or RStudio notebooks in DataHub, bypassing the complexities of Git that would otherwise create barriers to entry.
  • Install DataHub Link Generator extension in your chrome browser. If you need further assistance with the plugin installation, please check the following documentation.

Step 3 - Generate shareable link(s)

To generate a shareable link,

  • Access the notebook in the GitHub repository.
  • Generate a shareable link by following the step-by-step instructions in this documentation.
  • Share the link with students through tools such as bcourses, Ed discussions etc..

Students Getting Started

This guide provides an overview of the tasks you will need to complete before the semester begins, during the semester, and at the end of the semester. This high-level information will help you navigate your coursework effectively using the UC Berkeley DataHub.

Before the Semester Begins

1. Browser and Internet Check

  • Browser Compatibility: Ensure you are using a compatible web browser (Chrome, Firefox, Safari) that is updated to the latest version. 
  • Internet Connection: Verify you have a stable internet connection to access DataHub smoothly.

2. Account Setup and Access

  • Activate CalNet ID: Ensure your CalNet ID is activated and functioning properly.
  • Access DataHub: Go to datahub.berkeley.edu and log in with your CalNet ID to verify you can access the platform. ( Allow bcourses to authenticate “DataHub is requesting access to your account”.)

3. Familiarize Yourself with DataHub

  • Overview: Review student resources to understand the varied features of DataHub. Interface 
  • Tour: Explore the DataHub interface, including the JupyterLab and RStudio environments.

During the Semester

1. Accessing Course Materials

  • Course Hub: Use nbgitpuller links shared by your course instructors to launch notebooks in DataHub 
  • Notebooks and Scripts: Open and work on Jupyter Notebooks, R scripts, or other files as provided by your instructor. 
  • To manage files in JupyterHub: To upload a file, click the "Upload" button in the DataHub interface and select the file from your local machine. To download a file, right-click on the file in the DataHub interface and select the "Download" option

2. Completing Assignments

  • Regular Use: Regularly log in to DataHub to complete assignments, run analyses, and work on projects. 
  • Save Work: Save your progress frequently. It's good practice to manually save your work as well. 
  • Check Storage Space: Delete unnecessary files; Regularly check the storage size of the home directories and back up content if you are exceeding 5-10 GB. You can do this by opening a Terminal, and executing `du -sh` 
  • Don’t Duplicate Shared Directory Content: If your course work requires shared directories where instructors are storing large datasets, don’t create a copy of the same files in your home directory. As a practice, always read data from the shared directories. 
  • Do Your Work in Sub Directories: Create a sub directory for each assignment and do your work there. Avoid working on assignments from the root directory as they may lead to data issues if done wrongly.

3. Collaboration and Sharing

  • Note: DataHub doesn’t offer collaboration(link is external)(link is external) tools that allow students to work with each other on the same notebook. We are testing the feature extensively and will roll it out when the known security and data corruption issues are solved. 
  • Instructor Feedback: Share your work with instructors or TAs for feedback by downloading and submitting your notebooks as required.

4. Troubleshooting

  • Restart Kernel/Server: Try restarting your kernel as a classic troubleshooting step to see if the error goes away. If the problem persists, restart your server
  • Having too many notebooks open on Datahub can cause issues. To check running processes and kill them follow instructions to kill process in Curriculum Guide
  • Support: Reach out to course TAs for technical help, and they will contact the DataHub staff if they are unable to resolve your issue.
  • What if I can’t access DataHub?
    • Ensure your CalNet ID is active and try logging in again. If the problem persists, inform your TA or check out this guide for additional help.
  • How do I install additional packages?
    • Use `!pip install package-name` in a Jupyter Notebook cell for Python packages, or `install.packages("package-name")` in the R console.
  • Can I use DataHub off-campus?
    • Yes, you can access DataHub from anywhere with an internet connection.
  • What should I do if I encounter a technical issue?
    • First, try restarting your kernel. If the issue persists, contact your course TA, and if they can’t resolve it they will reach out to DataHub staff.

End of the Semester

1. Backup Your Work

  • Backup coursework: Back up your notebooks and data to either your personal device or an external storage service like Google Drive, Dropbox. The data present in the course specific hubs will go away by the end of the semester.
  • User Home Directory Archiving: Files unused for 30 days will be archived and stored in a low cost storage. You will need to open a request with the DataHub team to retrieve your unused files.

2. Clean Up Your Workspace

  • Clean Up: Clean up your DataHub workspace by deleting unnecessary files and folders.
  • Feedback: Provide feedback on your experience with the DataHub team to help improve the service for future students.

Get Help

Service at a Glance

CategoryDetails
Service Availability24/7*
Initial Support Response TimeNo longer than 1 business day
Services and OfferingsDataHub is the campus JupyterLab environment, which utilizes cloud computing infrastructure to deploy scalable resources that enable users to interact remotely with a standardized, common computing environment.
EligibilityInstructors and Students
Data Protection LevelP3 - This platform stores or processes ​​FERPA-Protected Student Records. It has undergone a Security Risk Assessment (SRA) and rigorous security testing by Berkeley's MICS program.
Service LevelEnhanced - We help you adapt the system or service to your needs. We are available for one-on-one consultations and workshops, or to coordinate the appropriate resources to help you accomplish your goals.
Service OwnerBalaji Alwar (datahub-support@berkeley.edu)
RatesOngoing operations support is free of charge.

* Except during scheduled maintenance.