Thursday, March 12, 2015
New Apps Script features at Google I O—again!
Scripts in Google Docs
Many of you have told us that you want to be able to extend Google Docs just like Google Sheets, with custom menus, dialogs, and triggers. Starting today, you can do just that (plus custom sidebars, too). To learn more about Apps Script in Docs—including a couple of secret features that we can’t tell you about yet!—please tune into the live stream with me and Jonathan Rascher on Thursday at 3:30pm PT.Forms Service / Scripts in Google Forms
In response to another top request, you can now use the Forms Service to programmatically create and modify Google Forms, including triggers and a better way to respond to form submissions. (We’ve created a new 5-minute quickstart to get you going.) You can also extend the Google Forms editor with the same custom menus, dialogs, and sidebars as Google Docs. If you’re at I/O, learn how to build Forms with Apps Script by joining Eric Koleda and Matthew Ziegelbaum on Wednesday at 1:55pm PT.Drive Service
For those of you who use the DocsList Service to automate your Google Drive, a newer version is now available. Drive Service comes with new features like setting the owner of a file or folder or changing the sharing settings. We designed the new service from the ground up to make it easier to work with large numbers of files and also fixed a lot of bugs. If you’re at I/O, Arun Nagarajan and John McGowan will give you more insight into Drive integration on Thursday at 1:40pm PT.Faster HtmlService
At Google I/O 2012, we launched HtmlService to let you build custom user interfaces with secure client-side scripting. Starting today, you can enable an experimental version of the client-side sandbox that runs significantly faster in any browser that supports ECMAScript 5 strict mode.Improved Authorization Flow and API Console Integration
You’ve also told us that authorizing a script takes too many steps. Now, you can opt in to an experimental new authorization flow that requires fewer clicks. In addition, every script that uses the new flow automatically creates a project in the Google APIs Console. This makes it much easier to use Google APIs that aren’t built in to Apps Script. To upgrade a script to the new flow, select File > Upgrade authorization experience. If you’re at I/O, Arun Nagarajan and Christoph Schwab-Ganser will demonstrate the new flow in their session on using the YouTube Analytics API with Apps Script on Wednesday at 1:55pm PT.As you can see, we’ve been working hard to improve Apps Script for you. We hope you enjoy the new features!
Saurabh Gupta profile | twitter | blog As the product manager for Google Apps Script, Saurabh is responsible for Apps Script’s overall vision and direction. |
How to use Google Apps APIs to handle large files
Retrieving and Storing User Profile Entries
The User Profile API makes it easy to divide retrieval of all profile entries into multiple fetch operations. The one area to tweak for this call is the size of results that can reliably be returned within 10 seconds on Google App Engine. In our testing, 100 entries per page is about the right size. Once configured, we will retrieve the feed containing the first set of entries, parse the response, and persist the results to cache. If that feed contains a link to another page containing more entries, we queue up a task to handle the next page and repeat until all pages are processed:ContactQuery query = null;//if this is the first page, there is no next link. Construct initial page urlif (nextLink == null) {String domain = getDomain(loggedInEmailAddress);String initialLink = PROFILES_FEED + domain + PROJECTION_FULL;query = new ContactQuery(new URL(initialLink));query.setMaxResults(GAE_OPTIMAL_PAGE_SIZE);} else {query = new ContactQuery(new URL(nextLink));}query.setStringCustomParameter(TWO_LEGGED_OAUTH_PARAM, loggedInEmailAddress);//fetch next profile feed containing entriesProfileFeed feed = contactsService.query(query, ProfileFeed.class);List> currentMaps = (List >)memcacheService.get(memcacheKey); for(ProfileEntry entry:feed.getEntries()){//secret sauce: convert entry into csv column header/value mapcurrentMaps.add(converter.flatten(entry));}//store updated list of converted entry maps back into memcachememcacheService.put(memcacheKey, currentMaps);if(feed.getNextLink()!=null){//start task to get next page of entriestasksService.fetchUserProfilesPageTask(spreadsheetTitle, loggedInEmailAddress, feed.getNextLink().getHref(), memcacheKey);}else{//no more pages to retrieve, start task to publish csvtasksService.exportMapsToSpreadsheet(spreadsheetTitle,loggedInEmailAddress,memcacheKey);}}
Exporting Profiles as a Google Docs Spreadsheet
One of the trickiest obstacles to work around in this effort is generating the Spreadsheet file since GAE restricts the ability to write to the File System. The Spreadsheets Data API was one possibility we considered, but we ended up feeling it was a bit of overkill, having to first create the Spreadsheet using the Docs List API and then populate the Spreadsheet one record per request. This could have generated thousands of requests to populate the entire Spreadsheet. Instead, we leveraged an open source library to write csv file data directly to a byte array, and then sent the file data as the content of the newly created Docs List entry in a single request:Once completed, this method results in a newly created, populated Google Docs Spreadsheet viewable in the logged-in users Google Docs.public void saveRowsAsSpreadsheet(String spreadsheetTitle, String loggedInEmailAddress, String memcacheKey) {//get list of csv column header/value maps from cache:List> rows = (List >)memcacheService.get(memcacheKey); //secret sauce: convert csv maps into a byte arrraybyte[] csvBytes = converter.getCsvBytes(rows);SpreadsheetEntry newDocument = new SpreadsheetEntry();MediaByteArraySource fileSource = new MediaByteArraySource(csvBytes,"text/csv");MediaContent content = new MediaContent();content.setMediaSource(fileSource);content.setMimeType(new ContentType( "text/csv"));//add MIME-Typed byte array as content to SpreadsheetEntrynewDocument.setContent(content);//set title of SpreadsheetEntrynewDocument.setTitle(new PlainTextConstruct(docTitle));URL feedUri = new URL(new StringBuilder(DOCS_URL).append(?).append(TWO_LEGGED_OAUTH_PARAM).append(=).append(userEmail).toString());docsService.insert(feedUri,newDocument);//we are done, time to delete the data stored in cachememcacheService.delete(memcacheKey);}
Conclusion
Rich signatures for your domain using the Email Settings and the Profiles APIs
A recent addition to the Email Settings API allows domain administrators to use HTML-encoded strings when configuring the default signature for their users.
Updating all signatures to make them adopt the same visually appealing style sounds like a perfect task to automate, however we’d still need to collect various pieces of information for each user, such as phone number or job title, and the Email Settings API has no knowledge of them.
The Google Apps Profiles API provides exactly what we are looking for and in the rest of this article we’ll see how to have the two APIs interact to reach our goal.
Let’s assume we want our signatures to look like the one in the screenshot below, with a bold name, italic job title and clickable link for the email address. Of course you can edit the style as you like with a bit of HTML skills:
Python is the programming language of our choice for this small script and we use the Google Data APIs Python Client Library to send requests to the Email Settings and Profiles APIs.
The first few lines of the script import the required libraries and set the values of the credentials that will be used to authorize our requests. You can find the consumer key and secret for your domain in your Control Panel, under Advanced Tools - Manage OAuth domain key. Remember to replace the dummy values in the script below with yours before running it:
import gdata.apps.emailsettings.client
import gdata.contacts.client
# replace these values with yours
CONSUMER_KEY = mydomain.com
CONSUMER_SECRET = my_consumer_secret
company_name = ACME Inc.
admin_username = admin
We’ll use 2-legged OAuth as the authorization mechanism and set the administrator’s email address as the value of the
xoauth_requestor_id
parameter, identifying the user we are sending the requests on behalf of.The consumer key and secret plus the requestor id are the only parameters needed to create an OAuth token that we can pass to the Email Settings and Profiles clients:
# request a 2-legged OAuth token
requestor_id = admin_username + @ + CONSUMER_KEY
two_legged_oauth_token = gdata.gauth.TwoLeggedOAuthHmacToken(
CONSUMER_KEY, CONSUMER_SECRET, requestor_id)
# Email Settings API client
email_settings_client = gdata.apps.emailsettings.client.EmailSettingsClient(
domain=CONSUMER_KEY)
email_settings_client.auth_token = two_legged_oauth_token
# User Profiles API client
profiles_client = gdata.contacts.client.ContactsClient(
domain=CONSUMER_KEY)
profiles_client.auth_token = two_legged_oauth_token
Let’s define a class that generates the signatures for our users on the basis of a set of optional attributes (occupation, phone number, email, etc). This is the class you need to edit or extend if you want to change the style of the signatures for your domain. In the example below, the
HtmlSignature()
method simply concatenates some strings with hard-coded styling, but you may want to use a more elaborate templating system instead:# helper class used to build signatures
class SignatureBuilder(object):
def HtmlSignature(self):
signature = %s % self.name
if self.occupation:
signature += %s % self.occupation
if self.company:
signature += %s % self.company
signature += Email: <a href=mailto:%s>%s</a> - Phone: %s % (
self.email, self.email, self.phone_number)
return signature
def __init__(
self, name, company=, occupation=, email=, phone_number=):
self.name = name
self.company = company
self.occupation = occupation
self.email = email
self.phone_number = phone_number
Let’s use profiles_client to retrieve a feed containing all profiles for the domain. Each call to
GetProfilesFeed()
only returns a page of users, so we need to follow the next
links until we get all users:# get all user profiles for the domain
profiles = []
feed_uri = profiles_client.GetFeedUri(profiles)
while feed_uri:
feed = profiles_client.GetProfilesFeed(uri=feed_uri)
profiles.extend(feed.entry)
feed_uri = feed.FindNextLink()
At this point profiles will contain the list of users we want to process. For each of them, we instantiate a
SignatureBuilder
object and set its properties name
, company
, occupation
, email
and phone_number
with the data for that user.A call to the HtmlSignature() method of the SignatureBuilder instance will provide us with a properly formatted HTML-encoded signature.
# extract relevant pieces of data for each profile
for entry in profiles:
builder = SignatureBuilder(entry.name.full_name.text)
builder.company = company_name
if entry.occupation:
builder.occupation = entry.occupation.text
for email in entry.email:
if email.primary and email.primary == true:
builder.email = email.address
for number in entry.phone_number:
if number.primary and number.primary == true:
builder.phone_number = number.text
# build the signature
signature = builder.HtmlSignature()
The Email Settings API client exposes a method called
UpdateSignature
to set the signature for a target user. This methods accepts two parameters, the username of the user to be affected and a string containing the signature. We just built the latter, so we only need the retrieve the unique username that identifies each user and that can be easily inferred from the entry identifier returned by the Profiles API, as described in the code and the comment below.It is worth mentioning that you can also retrieve usernames with the Provisioning API, but for the sake of simplicity we’ll rely on this small hack:
# entry.id has the following structure:
# http://www.google.com/m8/feeds/profiles/domain/DOMAIN_NAME/full/USERNAME
# the username is the string that follows the last /
username = entry.id.text[entry.id.text.rfind(/)+1:]
It’s time to send the requests to the Email Settings API and update the signature:
# set the users signature using the Email Settings API
email_settings_client.UpdateSignature(username=username,
signature=signature)
For further details on what can be accomplished with the Google Apps APIs, please check our documentation and don’t hesitate to reach out to us on our forums if you have any questions.
Claudio Cherubino profile | twitter | blog Claudio is a Developer Programs Engineer working on Google Apps APIs and the Google Apps Marketplace. Prior to Google, he worked as software developer, technology evangelist, community manager, consultant, technical translator and has contributed to many open-source projects, including MySQL, PHP, Wordpress, Songbird and Project Voldemort. |
Wednesday, March 11, 2015
Building a Rails based app for Google Apps Marketplace
Editors note: This is a guest post by Benjamin Coe. Benjamin shares tips and best practices on using Ruby on Rails for integrating with Google Apps and launching on the Marketplace. — Arun Nagarajan
- Replacing OpenID with OAuth 2.0 for Single-Sign-On.
- Replacing 2-legged OAuth with OAuth 2.0 Service Accounts, for delegated account access.
- Releasing a Gmail Contextual Gadget that worked within these new authentication paradigms.
OAuth 2.0 for SSO
In the revamped Google Apps Marketplace, OAuth 2.0 replaces OpenID for facilitating Single-Sign-On. The flow is as follows:- OAuth 2.0 credentials are created in the Cloud Console, within the same project that has the Google Apps Marketplace SDK enabled.
- When accessing your application, a user is put through the standard OAuth 2.0 authentication flow using these credentials.
- If the user has the Google Apps Marketplace App installed they will be logged directly into your application, skipping the authorization step.
Rails.application.config.middleware.use OmniAuth::Builder do
provider :google_oauth2, ENV["GAM_OAUTH_KEY"], ENV["GAM_OAUTH_SECRET"]
end
Yesware already had a Google OAuth 2.0 authentication strategy, so we opted to subclass the Google OAuth 2.0 OmniAuth Strategy. This allowed us to continue supporting our existing OAuth 2.0 credentials, while adding support for Google Apps Marketplace SSO. Our subclassed strategy looked like this:
# Subclass the GoogleOauth2 Omniauth strategy for
# Google Apps Marketplace V2 SSO.
module OmniAuth
module Strategies
class GoogleAppsMarketplace < OmniAuth::Strategies::GoogleOauth2
option :name, google_apps_marketplace
end
end
end
Rails.application.config.middleware.use OmniAuth::Builder do
provider :google_oauth2, ENV["OAUTH_KEY"],
ENV["OAUTH_SECRET"],
{:scope => ENV["OAUTH_SCOPE"]}
provider :google_apps_marketplace, ENV["GAM_OAUTH_KEY"],
ENV["GAM_OAUTH_SECRET"],
{ :scope => ENV["GAM_OAUTH_SCOPE"],
:access_type => online }end
Note that :access_type is set to online. This is necessary to prevent the authorization prompt from being presented to a SSO user. Omniauth defaults to an :access_type of offline.
Thats all it takes. With this OmniAuth strategy in place, when a domain administrator installs your application SSO will be available across the domain.
OAuth 2.0 Service Accounts
To support Yeswares reminder functionality, we needed offline access to a users email account. In the past, this functionality was supported through 2-legged OAuth. In the new Google Apps Marketplace paradigm, OAuth 2.0 Service Accounts are the replacement.- In the Cloud Console, generate a private key for the OAuth 2.0 Service Account associated with your Google Apps Marketplace project.
- Download the .p12 private key generated.
- Place this key somewhere that will be accessible by your production servers, e.g., a certificates folder in your codebase.
Gmail.connect!(:xoauth, ben@example.com, {Using the new Service Account Based Approach, it was as follows:
token: authentication.token,
secret: authentication.secret,
consumer_key: google.key,
consumer_secret: google.secret,
read_only: true
})
key = Google::APIClient::PKCS12.load_key(With OAuth 2.0 Service Accounts, the underlying libraries we used to interact with Gmail remained the same. There were simply a few extra steps necessary to obtain an access token.
google_apps.service.p12path, # this is a constant value Google uses
# to password protect the key.
notasecret
)service_account = Google::APIClient::JWTAsserter.new(
google_apps.service.email,
https://mail.google.com/,
key
)client = Google::APIClient.new(
:application_name => APPLICATION_NAME,
:version => APPLICATION_VERSION
).tap do |client|
client.authorization = service_account.authorize(ben@example.com)end
Google.connect!(:xoauth2, ben@example.com, {
:oauth2_token => client.authorization.access_token,
})
Contextual Gadgets and SSO
Yesware provides a Gmail Contextual Gadget, for scheduling email reminders. To facilitate this, its necessary that the gadget interact with a users email account. To make this a reality, we needed to implement SSO through our contextual gadget. Google provides great reading material on this topic. However, the approach outlined concentrates on the deprecated OpenID-based SSO approach. We used a slightly modified approach.Rather than OpenID, we used OAuth 2.0 for associating the opensocial_viewer_id with a user. To do this, we needed to modify our OmniAuth strategy to store the opensocial_viewer_id during authentication:
# Subclass the GoogleOauth2 Omniauth strategy forOnce an opensocial_viewer_id was connected to a Yesware user, we could securely make API calls from our contextual gadget. To cut down on the ritual surrounding this, we wrote a Devise Google Apps OpenSocial Strategy for authenticating the OpenSocal signed requests.
# Google Apps Marketplace V2 SSO.
module OmniAuth
module Strategies
class GoogleAppsMarketplace < OmniAuth::Strategies::GoogleOauth2
option :name, google_apps_marketplace
def request_phase
# Store the opensocial_viewer_id in the session.
# this allows us to bind the Google Apps contextual
# gadget to a user account.
if request.params[opensocial_viewer_id]
session[:opensocial_viewer_id] = request.params[opensocial_viewer_id]
end
super
end
end
end
end
Now Go Forth
Once we figured out all the moving parts, we were able to use mostly off the shelf mature libraries for building our Google Apps Marketplace Integration. I hope that this retrospective look at our development process helps other Rails developers hit the ground running even faster than we did.Benjamin Coe profile Benjamin Coe cofounded the email productivity company Attachments.me, which was acquired by Yesware, Inc., in 2013. Before starting his own company, Ben was an engineer at FreshBooks, the world’s #1 accounting solution. Ben’s in his element when writing scalable cloud-based infrastructure, and loves reflecting on the thought-process that goes into this. A rock-climber, amateur musician, and bagel aficionado, Ben can be found roaming the streets of San Francisco. ben@yesware.com — https://github.com/bcoe— @benjamincoe |
Supporting the growing Google Drive developer community Google Drive Workshops
Since the public unveiling of the Google Drive SDK in April, companies like Lucidchart or HelloFax have built powerful, slick, useful Google Drive apps, and many more companies are launching compelling integrations every day. During this time, our developer community — especially on Stack Overflow — has grown substantially.
To help support our growing developer community and all the interest in integrating with Google Drive, we’re starting a series of Google Drive developer workshops. For the inaugural event, we are hosting several companies — including Shutterfly, Fedex, Autodesk, Twisted Wave, 1DollarScan and Manilla — to participate in a two-day workshop this week at the Googleplex in Mountain View, California.
During this workshop, Google engineers will be on hand to assist attendees with various parts of their Google Drive integration: things like design and implementation of features, authorization flow, and Android integration. Companies have shown that the Google Drive SDK allows for deep integration in just a couple days and we really hope that attendees of this workshop will enjoy a similar experience. Tune back in later this week to find out more about what we learned and accomplished in our workshop.
If you are interested in attending similar Google Drive workshops near you or if you want to contact the Google Drive team about a potential integration with your product, let us know.
Nicolas Garnier Google+ | Twitter Nicolas Garnier joined Google’s Developer Relations in 2008 and lives in Zurich. He is a Developer Advocate for Google Drive and Google Apps. Nicolas is also the lead engineer for the OAuth 2.0 Playground. |
Retiring the Email Migration API
Posted by Wesley Chun, Developer Advocate, Google Apps
Last summer, we launched the new Gmail API, giving developers more flexible, powerful, and higher-level access to programmatic email management, not to mention improved performance. Since then, it has been expanded to replace the Google Apps Admin SDKs Email Migration API (EMAPI v2). Going forward, we recommend developers integrate with the Gmail API.
EMAPI v2 will be turned down on November 1, 2015, so you should switch to the Gmail API soon. To aid you with this effort, weve put together a developer’s guide to help you migrate from EMAPI v2 to the Gmail API. Before you do that, here’s your final reminder to not forget about these deprecations including EMAPI v1, which are coming even sooner (April 20, 2015).
Help us shape Google Developers Live for Google Drive
The week before Google I/O, we launched Google Developers Live, a new online channel to connect us with developers from all around the world, all year round.
Google Developers Live features interactive broadcasts about many different products and with many different formats. For Google Drive and Apps Script alone, we have aired app reviews, presentations about Google Drive and Apps Script, question and answer sessions, and a doc feedback session.
We are really interested in knowing from you about your favorite shows. Which types of event would you like to see more of in the future? Are you more interested in introductory material such as getting started tutorials, or more advanced topics?
Please share your feedback with us by adding a comment to this post or by reaching out to us on Google+.
Remember, we go live every Monday and Thursday and our complete schedule can be found at https://developers.google.com/live/drive. See all of you on Google Developers Live!
Claudio Cherubino profile | twitter | blog Claudio is an engineer in the Google Drive Developer Relations team. Prior to Google, he worked as software developer, technology evangelist, community manager, consultant, technical translator and has contributed to many open-source projects. His current interests include Google APIs, new technologies and coffee. |
Tuesday, March 10, 2015
Last Call for Google I O Google Apps Challenge
The last challenge, starting on March 29th at 4:00 P.M. PDT, will be based on Google Apps and we’d encourage all eligible developers to participate. To be sure that you’re prepared for the challenge, we have indicated a prerequisite to read up on Google Apps Script.
For more information on the contest rules and timeline, please see the Google Code Blog and the Contest Site.
Posted by Ryan Boyd, Google Apps Team
Want to weigh in on this topic? Discuss on Buzz
Failure logs can earn and cost money
Main.WriteLog("/mistake/fails.txt", "item nr. " + itemnr, false);
This line is worth millions in both ways:
1. When you develop application, this line helps me see the problems when something is not working. I put it on several places in my code and examine the results in the named text file. Sometimes it makes me see the problem in seconds that would take 3 days to notice otherwise.
2. When website is online, this line costs time. Especially if youre amateur enough to leave it there and run it on big dataset so it goes through several times. Every hit means writing to disk. And writing to disk is a costly operation.
So you can imagine that this line is about to change to write to database. :-) Or perhaps having additional option to write to disk or database. Sometimes the function behind happens to be rather useful for other reasons as well.
Documents List API Best Practices Handling errors and exponential backoff
This new documentation details all errors and the scenarios that cause them. We strongly recommend that both new and advanced Google Documents List API developers read the section thoroughly.
An important technique described in the new docs is exponential backoff. Exponential backoff helps clients to automatically retry requests that fail for intermittent reasons. For example, an application might make too many requests to the API in a short period of time, resulting in HTTP 503 responses. In cases like this, it makes sense for API clients to automatically retry the requests until they succeed.
Exponential backoff can be implemented in all of the Google Data API client libraries. An example in Python follows:
import random
import time
def GetResourcesWithExponentialBackoff(client):
"""Gets all of the resources for the authorized user
Args:
client: gdata.docs.client.DocsClient authorized for a user.
Returns:
gdata.docs.data.ResourceFeed representing Resources found in request.
"""
for n in range(0, 5):
try:
response = client.GetResources()
return response
except:
time.sleep((2 ** n) + (random.randint(0, 1000) / 1000))
print "There has been an error, the request never succeeded."
return None
We strongly recommend developers take about 30 minutes and update applications to use exponential backoff. For help doing this, please read the documentation and post in the forum if you have any questions.
Vic Fryzel profile | twitter | blog Vic is a Google engineer and open source software developer in the United States. His interests are generally in the artificial intelligence domain, specifically regarding neural networks and machine learning. Hes an amateur robotics and embedded systems engineer, and also maintains a significant interest in the security of software systems and engineering processes behind large software projects. |
Golden Nuggets from the Innovation Games Summit
1. Many of the attendees at my Silence of Agile talk were trained and experienced facilitators and during our discussions offered two specific tips that I plan on trying at one of our teams next retrospectives.
- The first tip was to change the voting so that people vote not only for their top 3 ideas but also their bottom 3. When tallying the results use the net votes (top minus bottom) as your top items to work on for that period.
- The second tip involves changing the silent writing portion of the retrospective. Instead of writing each individual idea on one post-it, ask each team member to write all their ideas on one paper in a list format. Once everyone has written their list, pass it to the person on your left. Each person then reads their neighbours list and adds to it. Keep passing the lists to the left until everyone has read and contributed to all the lists. Finally, put the individual items on your board and prioritize them as usual.
3. As a facilitator, it is important to trust the Innovation Game and let the group create their own process and flow within the context of the game. Gerry Kirk was my facilitation partner for the Budget Games and I watched (sometimes in trepidation) as he deftly used good questions to nudge the group along rather than directing the flow. Their process was sometimes a little scary as they tried to work within the game to come to decisions. However, in the end it was their process and their results. The game did its job to provide structure and Gerrys gentle questioning prodded them to a good result much more effectively than a directive approach would have.
4. A new podcast source! Look for Jack Dorsey and others on Standford Universitys Entrepreneurship Corner.
5. A new book that was highly recommended by Mr. Holdorf: The Radical Leap Re-Energized by Steve Farber.
6. Two Ss. These arent agile tips, but useful nonetheless
- Have sinus trouble when flying? A client of mine recommended Sinusalia by Boiron. Take two pills before you get on the plane and then every two hours during your flight - they are magical.
- Need music for your training sessions, workout sessions, relaxing at home, etc? Try the Songza app. It has lots of curated music based on themes and also comes with a pretty neat UX experience that suggests musical themes based on the time of day.
Monday, March 9, 2015
How we built a Social Network for Productivity on Top of Google Docs Calendar Contacts and Reader
Editor’s note: This is a guest post by Martin Böhringer, Co-Founder and CEO of Hojoki.
-- Steve Bazyl
Hojoki integrate productivity cloud apps into one newsfeed and enables sharing and discussions on top of the feed. We’ve integrated 17 apps now and counting, so it’s safe to say that we’re API addicts. Now its Time to share with you what we learned about the Google Apps APIs!
The goal: a newsfeed
Our initial reason for building Hojoki was because of the fragmentation we experience in all of our cloud apps. And all those emails. Still, there was this feeling of “I don’t know what’s going on” in our distributed teamwork. So we decided to build something like a Google+ where streams get automatically filled by activities in the apps you use.
This leads to a comprehensive stream of everything that’s going on in your team combined with comments and microblogging. You can organize your stream into workspaces, which are basically places for discussions and collaboration with your team.
Information needs
To build this, we first need some kind of information on recent events. As we wanted to be able to aggregate similar activities and to provide a search, as well as splitting up the stream in workspaces, we also had to be able to sort events as unique objects like files, calendar entries and contacts.
Further, it’s crucial to not only know what has changed, but who did it. So providing unique identities is important for building federated feeds.
Google APIs
Google’s APIs share some basic architecture and structure, described in their Google Data Protocol. Based on that, application-specific APIs provide access to the application’s data. What we use is the following:
- Google Docs: Google Documents List API version 3.0
- Google Calendar: Google Calendar API v2
Note: there is a new API v3 for Google Calendar - Google Contacts: Google Contacts API v2
The basic call for Google Contacts for example looks like this:
https://www.google.com/m8/feeds/contacts/default/full
This responds with a complete list of your contacts. Once we have this list all we have to do is to ask for the delta to our existing knowledge. For such use cases, Google’s APIs support query parameters as well as sorting parameters. So we can set “orderby” to “lastmodified” as well as “updated-min” to the timestamp of our last call. This way we are able to keep the traffic low and get quick results by only asking for things we might have missed.
If you want to develop using those APIs you should definitely have a look at the SDKs for them. We used the Google Docs SDK for an early prototype and loved it. Today, Hojoki uses its own generic connection handler for all our integrated systems so we don’t leverage the SDKs anymore.
Real world problems
If you’re into API development, you’ve probably already realized that our information needs don’t fit into many of the APIs out there. Most of the APIs are object centric. They can tell you what objects are included in a certain folder, but they can’t tell you which object in this folder has been changed recently. They just aren’t built with newsfeeds in mind.
Google Apps APIs support most of our information needs. Complete support of OAuth and very responsive APIs definitely make our lives easier.
However, the APIs are not built with Hojoki-like newsfeeds in mind. For example, ETags may change even if nothing happened to an object because of asynchronous processing on Google’s side (see Google’s comment on this). For us this means that, once we detect an altered ETag, in some cases we still have to check based on our existing data if there really have been relevant activities. Furthermore, we often have trouble with missing actors in our activities. For example, up to now we know when somebody changed a calendar event, but there is no way to find out who this was.
Another issue is the classification of updates. Google’s APIs tell us that something changed with an object. But to build a nice newsfeed you also want to know what exactly has been changed. So you’re looking for a verb like created, updated, shared, commented, moved or deleted. While Hojoki calls itself an aggregator for activities, technically we’re primarily an activity detector.
Final solution
You can think of Hojoki as a multi-layer platform. First of all, we try to get a complete overview on your meta-data of the connected app. In Google Docs, this means files and collections, and we retrieve URI and name as well as some additional information (not the content itself). This information fills a graph-based data storage (we use RDF, read more about it here).
At the moment, we subscribe to events in the integrated apps. If detected, they create a changeset for the existing data graph. This changeset is an activity for our newsfeed and related to the object representation. This allows us to provide a very flexible aggregation and filtering on the client side. See the following screenshot. You can filter the stream for a certain collection (“Analytics”) or only for the file history or for the Hojoki workspace where this file is added (“Hojoki Marketing”).
What’s really important in terms of such heavy API processing is to use asynchronous calls. We use the great Open Source project async-http-client for this task.
A wishlist
When I wrote that “we subscribe to events” this is a very nice euphemism for “we’re polling every 30s to see if something changed”. This is not really optimal and we’d love to change it. If Google Apps APIs would support a feed of user events modelled in a common standard like ActivityStrea.ms, combined with reliable ETags and maybe even a push API (e.g. Webhooks) this would also make life easier for lots of developers syncing their local files with Google and help to reduce traffic on both sides.
Martin Böhringer Martin is Co-Founder and CEO of Hojoki. Hojoki integrates Google Docs, Calendar, Contacts and Reader next to other apps like Dropbox and Evernote into one newsfeed, showing changes by co-workers in one central place. Martin holds a PhD in information systems and likes to perform on stage at Tech Events. |
Agile Chartering an Agile Documentation Alternative
"A document isn’t the only vehicle for expressing or transferring good thinking and ideas."Recently I coached a team that was converting an application from VB6 to VB.Net. One of the challenges of the project was to agree upon and define the rules for when to fix the old code, when to just get it working, when to re-write whole sections, etc. One method to gain agreement on this approach could be to schedule a series of meetings and write up the conversion rules and strategies in the project charter document. Here is what we did instead:
Using a variation of the silent brainstorming technique we spent about 20 minutes creating a light weight conversion manifesto we could all agree upon. As you can see, we had some fun when we named the groups. The manifesto was documented using an easel pad, post-its and sharpies. We placed it on the wall in the team room and referenced it often throughout the project. Situations we could not have documented up front surfaced during the project and were discussed quickly while referencing the post-its. This light weight manifesto served as a great vehicle for expressing our good thinking and ideas.
A meeting and a word document could have accomplished this as well, but not in the same amount of time, with the same effectiveness, or with the same mirth. In addition this is a great team building experience for teams that already have enough experience with long meetings and longer documents.
For more ideas on how to streamline your project chartering, take a look at this new book by Diana Larsen and Ainsley Nies: Liftoff: Launching Agile Teams and Projects
Image Via ThunderBoxRoad.com |
Subscribe to Winnipeg Agilist by Email
Going Deeper More Integration Examples Available in our Developer Docs
To help you discover ways to integrate your own applications with Google Apps, we added a new section to our developer docs showcasing examples from popular applications in the marketplace. Some examples, like BatchBook’s streamlined sign-up process for users, show simple ways applications can help users get started quicker.
Apps can help users be more productive by displaying information and allowing them to take action right within an e-mail message. Harvest, a time tracking application, allows users to complete their timesheets in gmail when they receive their weekly reminders.
You’ll also find examples for collaboration and document sharing, such as Manymoon’s seamless integration with Google Docs. This makes it easy for users to share information with teams or attach relevant docs to tasks.
You can read more about these and other integrations, or try out the apps yourself by going to the Google Apps Marketplace. Of course these are just a few of many ways to make apps easier for users, and we look forward to seeing even more creative ideas from developers.
Posted by Steven Bazyl, Google Apps Marketplace Team
P.S. If you want to learn about the APIs which enable these deep integrations, meet with fellow developers and ask questions of Googlers, come to our Google Apps and Apps Marketplace Hackathon at the Google Mountain View campus on August 24th!
Programmatically Managing Charts in Spreadsheets
Editor’s Note: Kevin Winter is a guest author from the AdWords API Developer Relations team. He implemented these features in his 20% time. - Jan Kleinert
Google Spreadsheets provides powerful charting functionality to let you analyze your data many different ways. We recently added the ability to programmatically create, modify and delete Spreadsheet charts using Google Apps Script.
What are Embedded Charts?
Charts that a user creates in a Google Spreadsheet are called Embedded Charts. Apps Script can be used to manage these types of Spreadsheet Charts. Each EmbeddedChart
belongs to a Sheet
. Embedded charts are named this way because they are embedded onto a Sheet
within the spreadsheet. Embedded charts align to the upper left hand corner of the specified column and row in the spreadsheet.
An Example of an Embedded Spreadsheet Chart
Let’s say I want to track my gas mileage using Google Spreadsheets. First, I create a Spreadsheet, and set up columns and formulas. Next, I add a form interface to make it easy to add new entries and a chart to visualize my mileage. Now I can start recording data - but each time I add enough entries to go past the ranges the chart uses, I need to manually increase them. How about we use some Apps Script magic to solve this?
The script below iterates through all charts on this sheet and determines if any of the ranges need to be expanded (i.e. if there are more rows with data to display). It then updates the title, builds the new EmbeddedChart
object and saves it to the sheet. It could also add a menu interface or a trigger to execute this periodically or when the spreadsheet is edited.
function expandCharts() {
var sheet = SpreadsheetApp.getActiveSheet()
// Get a list of all charts on this Sheet.
var charts = sheet.getCharts();
for (var i in charts) {
var chart = charts[i];
var ranges = chart.getRanges();
// Returns an EmbeddedChartBuilder with this chart’s settings.
var builder = chart.modify();
for (var j in ranges) {
var range = ranges[j];
// rangeShouldExpand_ is defined later.
if (rangeShouldExpand_(range)) {
// Removes the old range and substitutes the new one.
builder.removeRange(range);
var newRange = expandRange_(range);
builder.addRange(newRange);
}
}
// Update title.
builder.setOption(title, Last updated + new Date().toString());
// Must be called to save changes.
sheet.updateChart(builder.build());
}
}
function rangeShouldExpand_(range) {
var s = range.getSheet();
var numColumns = range.getNumColumns()
var values = s.getSheetValues(range.getLastRow(),
range.getColumn(),
2,
numColumns);
for (var i = 0; i < numColumns; i++) {
// If the next row has the same pattern of values,
// it’s probably the same type of data and should be expanded.
if (!values[0][i] && !values[1][i]
|| !!values[0][i] && !!values[1][i]) {
continue;
} else {
return false;
}
}
return true;
}
function expandRange_(range) {
var s = range.getSheet()
var startRow = range.getRow();
var startCol = range.getColumn();
var numRows = range.getNumRows();
var numCols = range.getNumColumns();
while (rangeShouldExpand_(range)) {
numRows++;
range = s.getRange(startRow, startCol, numRows, numCols);
}
return range;
}
Creating New Charts in Spreadsheets
What if you wanted to create a new chart from scratch? You can do that too!
var sheet = SpreadsheetApp.getActiveSheet();
var chart = sheet.newChart()
.setPosition(5, 6, 5, 5)
.setChartType(Charts.ChartType.AREA)
.addRange(sheet.getActiveRange())
.build();
sheet.insertChart(chart);
In the above code example, we’ve gotten a reference to an EmbeddedChartBuilder
, set its position within the sheet, change the chartType, add the currently selected range and insert the new chart.
Modifying Charts in Spreadsheets
The EmbeddedChartBuilder
allows you to modify the chart in a couple ways:
- Add/Remove the ranges this chart represents via
addRange
andremoveRange
. - Set options that modify how the chart will be rendered as well as change the chart type with
setOption
andsetChartType
. - Change where the chart will be displayed (the cell and cell offset of the chart container) via
setPosition
.
Embedded charts use more options than regular Apps Script charts, so options are handled slightly differently. Developers can pass a dotted field path to change options. For example, to change the animation duration, you can do this:
builder.setOption("animation.duration", 1000);
Now that we’ve created a bunch of charts, your spreadsheet is probably pretty cluttered. Want to clear it and start afresh with charts? Just remove them all:
var sheet = SpreadsheetApp.getActiveSheet();
var charts = sheet.getCharts();
for (var i in charts) {
sheet.removeChart(charts[i]);
}
Take Your Charts with You
Just like standalone charts, you can use embedded charts elsewhere. You can add them to a UIApp or a sites page as well as sending them as an email attachment:
// Attach all charts from the current sheet to an email.
var charts = SpreadsheetApp.getActiveSheet().getCharts();
MailApp.sendEmail(
"recipient@example.com",
"Income Charts", // Subject
"Heres the latest income charts", // Content
{attachments: charts });
We hope you found this blog post useful. Enjoy editing embedded charts using Google Apps Script!
Kevin Winter profile Kevin is a Developer Programs Engineer in the AdWords API Team. He maintains the latest version of the Python and Java client libraries for the AdWords API. You can find him in the New York office. |
Sunday, March 8, 2015
Going beyond the basics putting Drive features to work at Lucidchart
Editor’s note: This is a guest post by Ben Dilts, CTO & Co-founder of Lucidchart.
-- Steve Bazyl
The release of Drive SDK allowing deep integration with Google Drive shows how serious Google is about making Drive a great platform for third parties to develop.
There are a handful of obvious ways to use the SDK, such as allowing your users to open files from Drive in your application, edit them, and save them back. Today, Id like to quickly cover some less-obvious uses of the Drive API that we’re using at Lucidchart.
Automated Backups
Applications have the ability to create new files on Google Drive. This is typically used for content created by applications. For example, an online painting application may save a new PNG or JPG to a users Drive account for later editing.
One feature that Lucidchart has long provided to its users is the ability to download their entire accounts content in a ZIP file, in case they (or we!) later mess up that data in some way. These backups can be restored quickly into a new folder by uploading the ZIP file back to our servers. (Note: we’ve never yet had to restore a user account this way, but we provided it because customers said it was important to them.)
The problem with this arrangement is that users have to remember to do regular backups, since theres no way for us to automatically force them to download a backup frequently and put it in a safe place. With Google Drive, we now have access to a reliable, redundant storage mechanism that we can push data to as often as we would like.
Lucidchart now provides automated backups of these ZIP files to Google Drive on a daily or weekly basis, using the API for creating new files on Drive.
Publishing Content
Another use for the files.create
call is to publish finished content. Lucidchart, like most applications, stores its editable files in a custom format. When a user completes a diagram or drawing, they often download it as a vector PDF, image, or Microsoft Visio file to share with others.
Lucidchart is now using the create file API to export content in any supported format directly to a users Google Drive account, making it easy to sync to multiple devices and later share those files.
Indexable Text
Google Drive cant automatically index content created by Lucidchart, or any other application that saves data in a custom format, for full-text search. However, applications now have the ability to explicitly provide HTML content to Google Drive that it can then index.
Indexable text provided to the Drive API is always interpreted as HTML, so it is important to escape HTML entities. And if your text is separated into distinct pieces (like the text in each shape in Lucidchart), you can improve full-text phrase searching by dividing your indexable text into one div or paragraph element per piece. Both the files.create
and files.update
calls provide the ability to set indexable text.
We hope that this overview helps other developers implement better integrations into the Google Drive environment. Integrating with Drive lets us provide and improve a lot of functionality that users have asked for, and makes accessing and using Lucidchart easier overall. We think this is a great result both for users and web application developers and urge you to check it out.
Ben Dilts Ben is the co-founder and CTO of Lucidchart, an online diagramming application built in HTML5 that features real-time collaboration. Previously, he was CTO of Zane Benefits where he led the development of an online health benefits administration platform which was featured on the front page of The Wall Street Journal. Ben holds a BS in Computer Science from Brigham Young University. |
Wednesday, March 4, 2015
Summer Daily Worksheets for Kindergarten Students Entering First Grade!
As such, Ive been putting together my summer vacation packet for them and I decided to go with a day by day packet! The worksheets included follow a day by day pattern:
- Math Monday
- Tell Me About Science Tuesday
- Writing Wednesday
- Sight Words Thursday
- Phonics Friday
The following skills are addressed:
Math Monday
Addition, Subtraction, Problem Solving, Numbers 0-20, Patterns, More and Less
Tell Me About Science Tuesday
Summer, Autumn, Five Senses, Seeds We Eat, Winter, Animals in Winter, Keeping Healthy, Spring, Butterfly Life Cycle
Writing Wednesday
Telling a story using first, next, and last
Sight Words Thursday
away, about, around, any, beside, boy, could, come, came, close, does, down, every, eat, first, from, funny, find, gave, going, help, how, inside, into, just, jump, knew, know, look, little, make, many, new, need, once, open, please, put, ride, run, said, says, them, there, thing, this, under, up, want, went, where, with, your, you
Phonics Friday
Beginning Sounds, Middle Sounds, Ending Sounds, Blends, Digraphs, CVC Word Families
Heres a preview of all of the pages... or you can check out the preview at TPT by clicking the picture!
Enjoy!