Embed code
it uh thank you it's good it's good to see that some people are still here and awake um
maybe just general um might my presentation will be more
on working with modern websites in google search
to not websites in general in search uh who here has website or works
on the website okay a bunch of alright who doesn't have a website
very few okay well it's good to see like how many people are actually awake if you like combined the two uh
some people are weak that's good we'd we can start with
that alright so uh and they there are lots of
misconceptions i think around moderate websites in java
script and and search and we
often see things like this javascript shorter se oh your website will disappear
people who make websites on line and work research they often say oh
it's javascript framework oh that doesn't work so when you look at angel or we
we see that a lot so is killing your se or or or e.
even worse this this hurts most because i i work at
google it's like even google doesn't support angular it's which
hi barracks but i uh some people have figured it out
and this is apparently the answer it's because of all of those
curly braces and angular so who here has using colour
alright some of you have you seen curly braces in your code yeah okay apparently this is because
of the curly braces but not not really so hopefully we can find out some details
right so i looked at some of the websites that you've probably also use
and i wanted to see how they actually work with search because
but my point of view of a lot of this is misconceptions but it also depends a bit on
on how you make your website so i checked one website that you might have tried out
uh i have side that's the age who has looked at this website before
done something they're alright if you if you so the good news is you put it
in this model friendly testing tool and it's it is it's pages mobile friendly
so that's that's a good start i guess uh the bad news is this this
white box here on the side that's actually how google sees the page
if it's mobile friendly because everybody sees nothing
um which probably isn't what's what to go see so i i will
have some information on what what kind of problems to look into
with the website like this uh as an exercise all of you can
maybe go home and look at this website and think about
what is actually the problem why is google nazi this why is like this website diamond
search so when you look at what this website in search it looks like this
we know that this website it as we know that there's a homepage there's some description on the homepage but we don't know
anything else about this website put this on the one hand it's a good start but it's not really what you want
so if you look for the schedule usage of site c. h. schedule like well
i don't know there's not much of the the swiss schedule here actually
find able in search and you all probably know where to go but uh
imagine if you had something like a go go home at home or
some other device where you wanted to see when is this session or when it's another session at the the first
you can't find that information because google doesn't know about the there is something is to to improve here
uh on the other hand if you look at some of the other have us well they do have a schedule you can
fine so it's not that google a it's a fest in general
uh it's more that something is broken on this website
in the way that it works on a phone or works on on a laptop but not for such a alright so
taking a quick step back a briefly about me i work
as a webmaster trends analysts that google here in switzerland
uh which means i work directly with the web search team i work with
engineers who were creating web search the crawling the indexing raking part
i've been doing that for over nine years now i'm not a javascript
developers so some of the things you might hear me say
might not be perfect from the javascript point of view but uh i look at
things more from the source and this is cool but on the side
uh you'll see some more about the presentation style um my
session has mostly two parts like what search leads how
web search works focusing mostly on google but a lot of this works for other search engines as well
uh and how rendering works which uh is how
google look search for example javascript heavy pages
how how we can figure out what the content is and then so on on fixing these issues and some useful tool
it's alright so this this is essentially the start of a
search engine this is something that a lot of search
engines do um that i think it's important to to understand when
you look at a website and how it's shown in
search like what what is actually happening here uh we we start off with a collection of knowing your l.
so these are your else that we've seen before that people have linked to that we found online
uh we can't scroll them all at once because i think the
internet would not uh if we try to do that um
i think last year we look at how many or else we know about and it's over a
hundred and thirty trillion euro so that's that's a lot of pages on the internet and
we'd like to call them all as quickly as possible that we have to prior i and that's done by the scheduler
the scheduler takes the known your else and that it goes often crawls um on the
internet uh what we've crawled on we take the content that we've received we
bring that content into our index and we try to render the pages like like a browser would try to do and then
we have everything in our index we take all the links to the new pages and try to follow those as well
and this is a cycle that continues on on uh
so what search leads to to actually function is
this list of u. r. l.'s we we have to know which your else actually look at so
for us it's really important and this is one of the things that
uh a lot of modern frameworks have been kind of not doing that
well uh we need to have unique and index of all your
so we need to have a u. r. l. that you can take and you can use w. get or corrode to to actually fetch the
content um as a bonus we we'd like to be able to actually
look at those or else so make sure that they're not black
uh that we get some content back and that we get some links to other pages within your website
so if we start with the dev site home page then we can find the schedule we can find information about the
speaker is about the location all of that five following links
so what's what's important for us any u. r. l.
uh it essentially if if you've worked with websites you you
know what are your uh looks like it's not that
complicated uh uh uh for the most part but it's it's
something that for for us in search is really
what what everything pivot around so that having a good clean your l. is important
uh we don't care so much if you have a half an a file name
or if you have parameters that's totally up to you uh what we
do care about is um if there's a euro with hash so if it
has a hash and then there's a like attack or page name
and that's something that we often drop a nother common problem that we see is
the saying you're all for different states uh this is something that uh i
normally the the modern frameworks don't do so much but uh we
saw a lot in the past where you would have
uh the the homepage actually running the javascript happen you can do everything on the home page and it doesn't change
your l. so in a case like that we don't have
the way of addressing specific content on the web so
uh and another thing to to watch out for usually less
of a problem with modern frameworks is uh irrelevant
your l. elements like session i. d.'s or user names and passwords we
we can index a lot of this if we find it
and uh getting it out of the index afterwards is sometimes
a bit of okay so for google since we
we've seen so many your l.'s uh we've also seen that a lot of them are actually duplicates
so we we fetch one your l. and it's the same as another one sometimes it's like with double load up in the beginning and
without up up up and saying contact for us it's a bit of
a pain because then we would have to index everything twice
whereas ideally we would recognise oh this is a sane we just fold them together
and we call that a canonical your l. um have any of you heard
about canonical is asian some of you okay so this is one way to
to recognise those who have looked into s. c. o. little bit already
uh one of those words that nobody uses otherwise um so we we take
all of these duplicate we pick one year l. as a representative
um and that's the one that we focus on you can tell us which
one you want you can give us hints about which one is
your preferred your how or we'll just yes and try to take but
ideally if you're consistent within your website within your web apps
then we will just use whatever you you get so that's really useful
i'm going back to your l.'s with the hash and um
one of the things that we've done i don't know maybe six seven years
ago uh back when when javascript was kind of fresher on the web
uh we work to create a in ajax crawling see where you could have your
rounds with the hash bangs or the hash and exclamation mark it down
and we would try to crawl a written version of that your l. and we would expect
your website to render the pages for so you could use javascript to run your site
uh with the hash training as an identifier and we will crawl they escaped fragment version
yeah and you need to give us the version that's actually already read so that worked for a while
and i think for for a lot of sites it it makes sense but in the meantime things
have progressed you can use the uh things like history
a. p. i. to provide real normal year olds
and still use the javascript framework on the side and we noticed that a lot of sites
set this up and they do it wrong or they have problems
with that and it's really hard to diagnose on your
side if you don't know where where the problem is actually
come so we've decided to stop recommending this setup
and instead just use normal your it's alright i am and
p. w. a. i assume most of you have heard
about amp until gays well some of u. s. are okay
great yeah so these are two technologies that from
or point of view i think a really exciting and uh a kind of future looking and need things you can do
with an um this presentation doesn't focus on these technologies we
try from the search point of view and not too
give any google technologies or in the google features any special treatment
so we treaties like like any other website at your house out of user p. w.
a. you can do that and uh still have everything index all okay so
how does google dot actually render the content so what we try to do a
when we recognise that the page uses javascript if it uses a javascript framework
we try to render it like we would with a normal model browser uh for the most part
we can do that fairly well but like is out with that have sites uh example
uh it's something that doesn't always work that wow so there is some
things to to watch out for uh things like promises service workers
for j. p. i. local storage uh these are not supported by google
bought a similarly yes six is something that's currently not supported
so if you're trying to use any of the more modern features it's important that you use only
fills where you can't to to support older browsers or to do feature detection you see
does this browser support for service workers and if so then i'll set up a service if
it doesn't support service workers and have some way to kind of gracefully handle that
so that the pages at least allow uh as nother point
which is kind of more unique to to google as
a search engine is we uh i'm rely on the
robots text file to let us know which parts
of your website we control so it's important for us that
we don't cause any issues on your server and
so we go to the robots text file the first time we look at the web site and see
which parts are we allowed to look at which parts are we not allowed to
work and if you use embedded your l.'s on your page javascript file
see as as if you use a guys and or block a robots text
that we can't actually see what what's happening that we can't fetch
the javascript file we don't know how to process that's that's a problem for
us uh search uh and another one that comes from being a search
engine and not a normal browser is we need to have links that use
a elements so that our normal links that point to other urals
rather than just on click and it's uh because google bought is kind
of a a simple browser it looks at the page and
it tries to find all these links in crawl through all of those it doesn't go off and click on everything on okay
so if you have some functionality that gets triggered from an on over or
from an on click handler like this group about probably won't discover that
by itself it doesn't know where to click or where you where you add additional functional um another thing
um so i have a lot of a special cases coming up for for situations where it
some of you might be running into these issues some of you might not um mobile first indexing
is something that we just recently announced and uh
for us this is a pretty big
shift uh for most websites it shouldn't be that much of a problem so it the the
difference here is in the past we've indexed and rendered pages like a desktop browser would
and now that more than half of our users are using smart phones
we think using a desktop browser to index content is a bit obsolete all
so we're switching to go about using a smart phone as well
and for indexing the content like you would see it on a smart for that
this is from from our point of view pretty big change and we
need to make sure that all of our other functionalities still continues working uh
for all of you who are using responsive web design were already creating
mobile first uh websites just doesn't really change anything so
this is kind of something where the the situations
where we see him most problems are where you have a mobile version of the website that's really
bad compared to a desktop for both you only have very limited functionality in a dust or in
a mobile web site and that could be a problem if the content is very limited on
a mobile web site and that could be a problem uh whereas if if it's already responsive if it's
the same content on both sides that's that's perfect so this is i think a good argument
to brain if someone comes to use as well we will just make a very simple mobile version
and people can click the view as desktop link if they want to see the full version
that's not going to work in the future um we tried to render pretty much all
pages that that we index uh but when we can't render the pages we
tend to fall back to the h. t. m. l. version book and uh for all of you who have use the single
page have a framework you will know that the the h. t. m. l. version of this page is very limited
essentially just loading the the javascript content and then the javascript actually hold
in all of the the content itself displays what's needed there
so for single page apps that's kind of a problem and that's something
that that we so what would have us side as well
where we index the the raw h. t. m. l. but uh we weren't able to actually see the full contents so we
had very limited view of what was actually on this what's uh one
thing that's also kind of unique with search engine is is uh
we he hugh is some some of the method that uh that
you have on your pages in the head of the pitch
so there are things that are fairly easy to specify in the head like title of the page
or the description which we show in the search results directly that's kind of the basic part
um what's less obvious is things like the the
robots might attack where you can tell us
this page should be indexed or shouldn't be indexed or things like you can tell us this
is the the german version this is the french version this is the english version
of this page i'm cool should recognise a connection between these different page and these are
sometimes hard to to test if uh if the page needs to be rendered
so this is one trick that you can do to kind of test this
you can take the page as you would see the browser say
d. m. l. and then load that again in a browser to see what the the
final rendered h. t. m. l. would look like and you can test to see
the head includes everything that you could um
one issue that's sometimes comes up uh
is specially with uh with web sites that use response of web design
is that you would sometimes take content you will hide it on the page you will
have something like a click to read more like this or you'll have a kind
of a tab it uh interface we can click on one pattern the second tab is
kind of hidden and then you click on the tab than it actually displays
uh this is sometimes a problem it's a so if the content is loaded by default
if it's already in h. t. m. l. uh just hidden by c. s. as
then we can still take that into account i'm in search we
tend to devalued out a little bit in the sense that we say well a user going to this
page will expect this content to be visible and if it's not visible by the fall then
maybe it's not the most important content on the stick but that's kind of are thinking they're a
them more problematic part is when there's an event that actually triggers the loading of discount
if you click on a tad and then it doesn't ajax cough and
it fetches a content from the server and then displays it
and we don't know all the user could click on this is like the for this bad or whatever
and that something will happen in the background to actually pull the new
content we we don't know about all that kind of a content
we cannot index at all um performance i i guess a lot
of you look into performances well it's uh one of the
i think more important topics on the web at after of
course making sure that everything works in in various browsers
uh but especially with modern frameworks we see it all the time that you open one h. t.
m. l. page added polls in like hundreds of different resources and it takes like ten twenty
thirty forty seconds actually load the page itself and for google this is a problem as well
because we have all of these billions of pages that we need to crawl under under
and if these pages take a minute or so to actually load all the content then
we would never get to actually indexing the the of all what uh
so i would generally recommend just following the normal performance best processes
uh reducing the number of requests bundling your javascript see as a as
uh be careful with a third party dependencies on you guys in particular
uh so if you have something that the relies on some third party a. p. i.
to get the content and then you display that content if that a. p.
i. is slow then of course your page is going to be slows well and
uh finally of course use fast servers uh uh content delivery network so
that at least static content is the liver very quickly write a rendering more rendering
so one way you can also control this on your side is if you
do the rendering yourself instead of relying on google bought you can render it
on your side there's some frameworks out there that i'm an issue that
a four year that provide some ways to to do that on your side
uh we recommend making sure that if you do rendering if you pretty
render your content to it for all users not just for google bought so
so technically doing i forgot about would be against the webmaster guideline
be showing different content to users and to google bought a from
a practical point of you there's some frameworks out there that
do that like a pretty render io uh for example that the
tricky part here is it's really hard to do but
because you send google one set of content you show everyone else a different set of content
and if something breaks interview that google sees you don't notice that none of your users
notice that nobody can tell you is something is broken so three rendering everything for
everyone usually makes the pages much faster and uh it make sure that everything is consistent
so this is the the free rendering part is often called isn't morphing javascript
uh on regular it's angular universal uh there are different ways to set this up this
is i i think a great way to handle this but it's sometimes very complicated
so i wouldn't see this as something that you could just switch on and then everything
will work you you really have to know what what you're doing their thing alright
so fixing old issues so if one of you fines uh the problems on the deaf
as the website like how do you handle uh from people's point of view
we try to keep it as simple as possible so you shouldn't need to tell google that you actually change something on
your website we should be able to figure that out or
sell so i put this in in three steps
which kind of over simplifies it because obviously finding the issue first is is
the hard part uh but i would recommend double checking non vocal
dependencies that even if you're doing this to fix an issue on google
make sure it still works whatever else you need your website
if you're doing things with things book if you're doing things with social media or other search engines or other platforms
make sure you don't break anything for that um then of
course bring up the clean version technically neural version and
then if you make your role changes so particularly from that the hash u. r. l.'s to clear your l.'s
then you need to make sure that you do reader it's
so redirects so uh for for google are important because
they let us know that we can forward of the information we know about the old pages to the new
page uh without a redirect what will happen is we'll see these are new your house and these urals
period we don't know what the connection it is so we essentially start those new pages
from zero and uh you you you start at the bottom of the search
results okay so with the redirect that's something that uh we can try
to connect um if you do redirects make sure you do them direct uh
if you do javascript redirect for fashion house being urals that's great
i'll show an interest special because that's something workable but
we'll get stuck again and uh there's um
trucks to kind of watch out for here which me since we have
documented it or help centre as well with regards to normal
side new so making sure that you list the old and the new year i'll so that we could we crawl down
i'm making sure that you've updated all of the internal references to the old new
you're also that we recognise that connection alright so let's look it's um
useful tools um who has you search council before a handful okay that's good
at least some of your still awake that's also a so research council
it is a it tool that we have from google which gives you aggregate
information on how your website is seen from goebbels point of view
uh so it can tell you which part of your website is kind
of broke and gives you information on links to your pages
what we've discovered when it shown in the search results for which
queries people are finding your website all of this information
around search is shown in such calls so if you have
a website i'd strongly recommend that you verify you
a website for such console that's easy to do not that
complicated and uh uh take a look at the the
settings and the errors that we have there for model websites
a lot of these things or a less critical
uh but sometimes things like the crawl rate is is
important for you to control overseeing that your servers
being overloaded with requests from google that's something to kind of look into what's actually happening here
if you have a web site that had is available in multiple countries and you can tell us this is one
is for germany this one's for switzerland this one is for france and that helps us to say well
we can show this website more to users in those countries yeah and finally i you mentioned
this last one here which is kind of the opposite problem of a lot of sight
in the sense that uh many people want to get into search first but uh
it sometimes you have to take stuff out of search quickly as well
uh so this happens every now and then that people index something that shouldn't be and uh maybe a bunch of
credit card statements maybe a bunch of address email list of
something that you accidentally made available on your website
yeah and group about is being busy and didn't oh this is great information to show to everyone in search
which maybe you don't want to have everyone see so in search console you can go
there and say i'd this specific your l. i want to have removed immediately
and within a day it'll be removed from so much so that gives you a
bit of time to figure out what actually happened here and why he
was a small this file on the server in a way that was publicly available
then kind of makes all of that but uh by being verified in search
console you're prepared for this and you can handle this very quickly uh in search
console there is the feature to um test how google renders a page
uh which we call fetches google and other other two options there one is to look at
the h. t. m. l. u. which is just the raw h. t. m. l.
uh the other is to look at the rendered version of the page uh which will be like google
bought as a desktop browser now or google but as a smart phone if you want and uh
it shows you what the page actually would look like to google it also list the the blocked him
better due oh so you're else that are blocked by robots text files which we can't access
and it gives you screenshot of what those pages would look like uh with with the block content
if we could crawl the block and it without so that's i think a really important tool
uh when you're working with modern web framework a psych
have files so i mention cycle files before uh
who here knows what type of file it's alright a bunch more let's get right so it's like
that file is basically a list of the your l.'s
on your website uh you tell google that
these are the your l.'s on your website that you know about which are important for you
yeah and we will take that list and try to make sure that all of these are actually shown in such a
so one one thing you can do an assignment file is specified they change date of the u. r. l.
and this is really helpful for us because then we can
recognise this page change we should go recall that now
and these other pages haven't changed in a long time and we don't have to worry
about them as much uh so updating the change date is really important for us
uh one thing that's useful with psycho files in search console
is that the yeah we show you errors that come from the side and file
and you can see the number of index your l.'s from the segments file so if
you send us all of your product pages or all of your event pages
then you can look at the sight of file for the product and c. o. ninety
percent of these got index uh that means these are actually doing pretty good
uh whereas if you look at the product pages new c. o. only like ten
of these pages got index and i submitted a hundred thousand of them and
you can kind of guess that's some kind of technical problem is probably still there that
you can resell alright so kind of coming to the hand i'm just as
the compress summary the most important part especially from other frameworks is to make sure
that you have clean your else uh so no hashes in the rounds
uh i really recommend understanding how google
dot renders pages i'm understanding which
functionality in in modern browsers aren't supported by by cool bought and then going off and
actually testing your pages are testing parts of your pages to to make sure that
they work and then hopefully everything will pan out and your pages will be index and
you'll be happy uh so it's i guess coming back to the deficit website
i don't know who which of you run strands what side but maybe if we if you leave it the
way it is now that gives others here a chance to kind of look at it and see
where could this problem long uh if you can't figure it out all
uh then you are of course welcome to contact me as well
uh we we can try to find some some answers that help you figure that's alright i don't know how much

Share this talk: 


Conference program

Keynote
Jean-Baptiste Clion, Coordinator DevFest Switzerland
26 nov. 2016 · 9:40 matin
How to convince organization to adopt a new technology
Daria Mühlethaler, Swisscom / Zürich, Switzerland
26 nov. 2016 · 10:14 matin
Q&A - How to convince organization to adopt a new technology
Daria Mühlethaler, Swisscom / Zürich, Switzerland
26 nov. 2016 · 10:38 matin
Animations for a better user experience
Lorica Claesson, Nordic Usability / Zürich, Switzerland
26 nov. 2016 · 11:01 matin
Q&A - Animations for a better user experience
Lorica Claesson, Nordic Usability / Zürich, Switzerland
26 nov. 2016 · 11:27 matin
Artificial Intelligence at Swisscom
Andreea Hossmann, Swisscom / Bern, Switzerland
26 nov. 2016 · 1:01 après-midi
Q&A - Artificial Intelligence at Swisscom
Andreea Hossmann, Swisscom / Bern, Switzerland
26 nov. 2016 · 1:29 après-midi
An introduction to TensorFlow
Mihaela Rosca, Google / London, England
26 nov. 2016 · 2:01 après-midi
Q&A - An introduction to TensorFlow
Mihaela Rosca, Google
26 nov. 2016 · 2:35 après-midi
Limbic system using Tensorflow
Gema Parreño Piqueras, Tetuan Valley / Madrid, Spain
26 nov. 2016 · 3:31 après-midi
Q&A - Limbic system using Tensorflow
Gema Parreño Piqueras, Tetuan Valley / Madrid, Spain
26 nov. 2016 · 4:04 après-midi
How Docker revolutionized the IT landscape
Vadim Bauer, 8gears AG / Zürich, Switzerland
26 nov. 2016 · 4:32 après-midi
Closing Remarks
Jacques Supcik, Professeur, Filière Télécommunications, Institut iSIS, HEFr
26 nov. 2016 · 5:11 après-midi
Rosie: clean use case framework
Jorge Barroso, Karumi / Madrid, Spain
27 nov. 2016 · 10:05 matin
Q&A - Rosie: clean use case framework
Jorge Barroso, Karumi / Madrid, Spain
27 nov. 2016 · 10:39 matin
The Firebase tier for your app
Matteo Bonifazi, Technogym / Cesena, Italy
27 nov. 2016 · 10:49 matin
Q&A - The Firebase tier for your app
Matteo Bonifazi, Technogym / Cesena, Italy
27 nov. 2016 · 11:32 matin
PERFMATTERS for Android
Hasan Hosgel, ImmobilienScout24 / Berlin, Germany
27 nov. 2016 · 11:45 matin
Q&A - PERFMATTERS for Android
Hasan Hosgel, ImmobilienScout24 / Berlin, Germany
27 nov. 2016 · 12:22 après-midi
Managing your online presence on Google Search
John Mueller, Google / Zürich, Switzerland
27 nov. 2016 · 1:29 après-midi
Q&A - Managing your online presence on Google Search
John Mueller, Google / Zürich, Switzerland
27 nov. 2016 · 2:02 après-midi
Design for Conversation
Henrik Vendelbo, The Digital Gap / Zurich, Switzerland
27 nov. 2016 · 2:30 après-midi
Q&A - Design for Conversation
Henrik Vendelbo, The Digital Gap / Zurich, Switzerland
27 nov. 2016 · 3:09 après-midi
Firebase with Angular 2 - the perfect match
Christoffer Noring, OVO Energy / London, England
27 nov. 2016 · 4:05 après-midi
Q&A - Firebase with Angular 2 - the perfect match
Christoffer Noring, OVO Energy / London, England
27 nov. 2016 · 4:33 après-midi
Wanna more fire? - Let's try polymerfire!
Sofiya Huts, JustAnswer / Lviv, Ukraine
27 nov. 2016 · 5 après-midi
Q&A - Wanna more fire? - Let's try polymerfire!
Sofiya Huts, JustAnswer / Lviv, Ukraine
27 nov. 2016 · 5:38 après-midi
Closing Remarks
Panel
27 nov. 2016 · 5:44 après-midi